A while back I decided to start tracking all the song playing data on musicxray.com. Looking for a lightweight solution that wouldn’t impact the performance of our application servers, I decided to send data directly from the browers to dynamodb.
The first step you will want to do is create a custom user that has one permission enabled for put operations only. This is important as you don’t want your data being deleted or viewed by the outside world. Basically we have a data bucket that we can send values into but can’t get them out without a different user with more permissions.
Once you have your dynamodb table set up you’ll want to start sending data to it. I choose a very simple schema. There are three fields on the table one id which serves as the hash and one date that serves as the range. Lastly we have a “data” field that stores a stringified blob of json.
On the front end my first attempt was to send the data in the main javascript loop. This turned out to work pretty well. However, I wasn’t satisfied with the UI responsiveness of the solution. I decided to employ web workers to really push the data tracking task to the background as quickly as possible.
I ran into some minor issues with loading the amazon sdk in the web worker. Hopefully the following helps save someone else a little bit of time.
So that’s my solution for tracking every second of play activity on our site in a low maintenance infinitely scaleable way. If you find this at all useful then please feel free to comment.