Serverless and Segment, a Match Made in Data Heaven

Trevor Heath

Trevor Heath

January 14 • 1 Min Read

In a world where data is king, it has become more and more critical that your company thinks about tracking and analytics the same way it thinks about frontend applications and backend infrastructure.


Data needs its own place to live, its own strategy, and it’s own infrastructure.

Our team has adopted a consistent, safe and modular way to track user, admin and programmatic events on the server side using Serverless, event driven architecture and Segment.io.  This convention allows us to move fast without requiring repeated changes to any of the code core to the business.  It’s a data team’s dream to iterate on tracking without redeploying any critical path backend or frontend code.  Let’s break down how Serverless and Segment work together to create a modular and flexible data pipeline.

SERVERLESS -

https://serverless.com/

A platform for event-driven architecture.

We use AWS lambda function as our foundation for Serverless.  With our event gateway we are able to listen for events emitted by any other server or frontend client.  These events can then trigger modular and self-contained lambda functions.  Event-driven architecture provides a safe and quick way to add functionality while limiting exposure to critical bugs.

SEGMENT

http://segment.io/


An analytics service that acts as a single API for all of your team’s data sources and destinations.

Segment allows you to track an event or identify a user with a consistent piece of code and send that information to pre-integrated data platforms where it can be best analyzed.  This flexibility allows development teams, marketers and data analysts to manipulate with data independently without blockers.

When these two conventions are combined, a new analytics superhero is created.  One with speed, agility, modularity and cost-efficiency.

Emit an event once from the API, frontend or another lambda function

Pass the appropriate data and event name.

Now when the event is triggered you can listen and invoke Serverless functions that will handle the event tracking utilizing Segment from your event gateway.  Broad events like database updates are good way to reduce the amount of code necessary in your API.  You can even handle this entirely from your event gateway by using the serverless-event-gateway-plugin which allows you to listen for webhooks from your database or other third party services (Database updates, chat apis, CRMs, ect.)

You can define these in your serverless.yml file but be aware there is a restriction to the number events you will be able add to it.  Check out this article by my colleague to learn about a micro-service architecture which allows you to split your code in separate services.

  1. Now handle the event and fire an event using Segment’s API! You even have a chance to query your database to collect relevant information before.

Wohooo! Now you will receive the events in Segment.  With a few clicks, you can easily configure Segment to pass that data on to the analytic tools you love to use!

This is a quick overview of the workflow we take to implement tracking with Serverless and Segment.  The power is really harnessed when all of the events you need have been emitted properly and you are taking advantage of database webhooks to capture changes to user information.  If the emitted events pass the proper contextual information like IDs it makes it really easy to add and change your Segment events without ever needing to dive back into the API handling your core business logic!

Work faster, worry less.


Tags
ServerlessDataDevelopmentTechnology