AWS RDS Proxy: A gamechanger for DB Connection Management
Technical Newsletter that covers broad topics including Microservices, Cloud, Machine Learning etc.
Welcome all!
We ended 2020 with a survey and our fam has spoken! Apart from continuing articles on Microservices, people want to see more content around Machine Learning as well as Cloud! I’ll make sure to include these two topics throughout this year and continue to learn together. Feel free to drop me a note if you have any comments/recommendations 👇
To mark the beginning of 2021, one of the highest asked topic was Microservices & Cloud and hence to start it off the perfect way, let’s learn more about one of the latest cloud offerings: AWS RDS Proxy.
Introduction
RDS Proxy is a great feature especially for those that are currently building on top of AWS and RDS by helping them manage connections to their DB. RDS Proxy is a middleman between your service and RDS endpoint. Usually, services connect directly with their RDS endpoint and then send commands to the cluster for their operations. With RDS Proxy, services connect to this proxy and in turn proxy connects to the DB.
This proxy helps us to worry only about connecting to itself and in turn handles entire network traffic to the DB. Proxy handles establishing, maintaining and even terminating the connections. It helps maintaining the connection pool essentially.
This is extremely helpful to those that try to connect multiple applications to their DB and end up hitting the maximum allowed limit to the number of connections. One of the major benefits of RDS proxy is that it helps keeping the actual connections to the DB low.
Another advantage is that it helps simplify the application logic because now you don’t have to worry about creating/maintaining the connections.
It’s also multi-usage i.e. you can use RDS proxy with Postgres, MySQL or Aurora DB.
How RDS Proxy Works?
Without RDS Proxy:
It’s just like the name suggest, if we don’t use the proxy, there’s a chance of running into exceptions when multiple applications are trying to connect to DB.
With RDS Proxy:
If your setup has multiple replicas or have different read-write replicas, then RDS proxy usually interacts with the writer instance in case of larger fleets.
In case of failover scenarios, RDS proxy will detect that writer instance is down and establishes a connection with the reader instance. If a similar thing happens for multiple availability zones, similarly proxy will establish a connection with read replica present in other zones.
Things to Keep in Mind
When you create your RDS proxy, a single proxy will be mapped to a single RDS instance as part of Aurora fleet. Usually, it’s the writer instance of RDS instance that gets 1:1 mapped to proxy.
When RDS proxy is setup, you get a new dedicated endpoint for your DB. You will continue to have previous endpoints for your RDS instance/read replicas of your DB.
During a failover scenario, you don’t need to worry about DB connection as proxy will take care of establishing connection with the read replica.
How does RDS Proxy work internally?
RDS internally relies on a couple of techniques to help with above scenarios of failover:
Multiplexing
Reusing the session for multiple transactions
Connection Borrowing
RDS proxy at times borrows a connection from connection pool
Pinning
This is a fallback behavior where RDS proxy isn’t able to determine if the existing connection can be re-used for other sessions. In these scenarios, proxy keeps the same connection alive for the same session until session has ended.
Conclusion
If you have a setup on AWS cloud and have always been worried about effectively managing DB connections, then RDS Proxy is a solution for all those questions.
Reference
For a more detailed read, analysis and setup of RDS proxy follow the official documentation here.
If you like the post, share and subscribe to the newsletter to stay up to date with tech/product musings.
(The contents of this blog are of my personal opinion and/or self-reading a bunch of articles and in no way influenced by my employer.)
Can you write example of a class library which does db connection management
First of all, I read you well. It helped me a lot.
I wonder if amazon rds proxy can split the read endpoint and the write endpoint. When I googled it, I saw an article saying that you need to use an external proxy tool, 'Heimdall Database Proxy'. So I'd like to know your opinion on this.