A Pyle of Stories - Page 8 of 9 - Tech, golf and life

Extending and Customizing the JWT from Cognito via AWS Lambda using Go

Posted on February 4, 2023by Benjamen Pyle

I’ve been working a lot lately with Cognito and User Pools in AWS as I’ve been wanting to migrate and existing app into a serverless Identity and Access provider. The promise of Cognito is this “Implement secure, frictionless customer identity and access management that scales” – AWS

Honestly there are so many identity providers out there. This article won’t go into the alternatives and other options out there but will specifically touch upon something that I know was a big question for me when I started with Cognito which was, “how can I customize the private claims in a token?”. So let’s discuss that a little further

As usual, if you want to skip straight to code, feel free to jump over to the repository here

Data

Unmarshalling a DynamoDB Map into a Go Struct

Posted on February 1, 2023by Benjamen Pyle

Short post on unmarshalling a DynamoDB Map into something strongly typed like a Go struct. If you want to jump straight to some code, here are the Github gists

Serverless

Handling Change with AWS Healthlake

Posted on January 30, 2023by Benjamen Pyle

One of the features that I am currently missing with AWS Healthlake is a proper “event-ing” framework. With DynamoDB you’ve got streams. With RDS you can use DMS. But with Healthlake there is no native change data capture mechanism.

Being that I’m only working on event driven architectures these days, I needed a way to be able to handle change. What I’m going to show you below is not “sanctioned” but it is 100% AWS native and continues with the Serverless theme. With that said, here’s the Github Repository if you just want to jump ahead. The CDK code will deploy a Healthlake instance @ $.27 / hr so please run cdk destroy npx ts-node bin/app.ts` when you are done

Serverless

Mapping AWS State Machine output to API Gateway response with VTL

Posted on January 27, 2023by Benjamen Pyle

This is a continuation of a previous article I wrote regarding zero code workflows creating Cognito users with Step Functions. Part of using State Machines with API Gateway is the dealing with the response and potentially VTL mapping

Goals of this article are to document some of the tips and things that I picked up along the way.

Serverless

Creating an AWS Cognito User with an Auto-Incrementing ID using AWS Step Functions

Posted on January 25, 2023by Benjamen Pyle

So there are a couple of interesting topics in here.

I’ve been really leaning into code-less workflows with AWS Step Functions and this State Machine has nothing but native SDK integrations which include
- DynamoDB (Put, Delete, Get)
- Cognito/User Pools (AdminCreateUser)
I’ve run into some legacy code that requires a Username to be a bigint and I don’t want to use an RDBMS so I’m using DynamoDB to generate one for me while also being “race condition” proof

As always, if you want to jump straight to the code, here is the Github repository

Serverless

API Gateway Base Path Mapping

Posted on January 16, 2023by Benjamen Pyle

AWS API Gateway is fantastic for sitting in front of AWS resources like load balancers and lambda functions and it’s also really great for setting nice domain boundaries around you application code. Let’s enhance the API Gateway experience a little more by levering Base Path Mapping

For instance, if you have boundary A with a set of REST operations and a boundary B with another set of REST operations you now end up with 2 API Gateways with their own FQDN to access those resources. But what if you want to have those separate, but also roll them up under a common domain name? This is where using API Gateway Custom Domains and Base Path mapping while turning off the default endpoints is so helpful. Article below is going to be pretty concise but also very specific to this problem and show how to use API Gateway Base Path Mapping with CDK

If you want to jump to the code, here is the Github repos

Data

Querying AWS Healthlake from Go

Posted on January 9, 2023by Benjamen Pyle

When working with Healthcare data when of the things that’s often mentioned or discussed is “Is your data interoperable?” As a developer and an architect, that’s a really loaded word to me because if I can expose my data over files, APIs or some consistent channel like TCP, then by definition my system is interoperable. Per my Mac dictionary “interoperable” is an adjective defined like this :: (of computer systems or software) able to exchange and make use of information ::

However where things get a little more nuanced is when the definition includes some common healthcare specific formats and more specifically HL7’s FHIR. So when you have this problem, there are certain tools that you need to use. There are several opensource solutions that you could select but when you are an AWS’ customer, you start with AWS first. And they just so happy to have a set of capabilities wrapped around a product called Healthlake.

Infrastructure

Handling “Poison Pill” Messages with AWS Kinesis and Lambdas

Posted on December 29, 2022by Benjamen Pyle

Queues and streams are fundamentally different in how they handle readers consuming their information.

With an SQS Queue you can have many consumers but generally one consumer will win reading the message and in the event of success the message is purged from the queue or upon failure that message is returned back to the queue. It technically doesn’t get deleted, yet the its visibility property is changed. Hence why the VisibilityTimeout on the queue matters. If your code processes messages in more time than that property then you are going to get messages that constantly get put back on the queue for retry.

Serverless

Creating an Async Integration with AWS Step Functions from API Gateway via CDK

Posted on December 17, 2022by Benjamen Pyle

I often have the scenario where there is a client which makes a request to an endpoint and they just want to make sure that payload was delivered but not necessarily concerned about the outcome. A pretty simple Async operation that happens over a quick Sync channel.

In the past, I’ve done my best either with a Lambda function to make sure it was so simple that it was incapable of failure. As I progressed further into that solution, I started using AWS Integrations to drop the payload off in an SQS Queue and then having a Lambda read that queue and then decide what to do.

Observability

Tracing HTTP Requests with Go and Datadog

Posted on December 13, 2022by Benjamen Pyle

Small follow up on the last post regarding tracing. I’m a huge fan of Event Driven systems or EDA (Event Driven Architecture) but sometimes you do need to make that synchronous HTTP request in order to fetch more data. Perhaps you are building a “saga” or sometimes events just published what happened and to whom it happened but not specifics about the actual event. For that you need to return back out and fetch more info.

When that happens, you’ll need to use a HTTP Client for making that request. And when doing so, it often sort of turns into a black hole, especially if you have multiple calls to make and you need to distinguish them. Enter again the Datadog libraries. With a simple wrapping of the client, when you make requests WithContext you will get a nicer and prettier display of what the span is. In the case below, I usually like to set the VERB that was requested in addition to the URL. Feel free to use/show whatever makes sense to you