User-Defined Database Functions (UDFs) as Microservices + creating a whole microservice cluster with pl/proxy

October 20–23
Level: Intermediate

While some people shun database functions as something that is hard to develop, test and debug I will show that the opposite is true - having a clearly designed API between your application and database not only makes all the above easier but also allows infinite scalability with full developer control and very clean and easy to understand model.

I will also show that testing database functions is 1:1 similar to testing microservices and should be approached exactly the same way.

And some testing is just not needed, as PostgreSQL itself can enforce many business rules by just defining them as constraints.

And I will also talk about one of the great things that came out of Skype's PostgreSQL team - pl/proxy- providing virtually unlimited scalability with no SPOFs

The original version of pl/proxy was invented and implemented at Skype about 20+ years ago. A little over two years later full rewrite - and splitting out pgbouncer- was published as pl/proxy version 2.0 on github.

The original target

The original, and successful, target was to give PostgreSQL a tool that allows functions infinite scalability via sharding inside the function.

This enabled an architecture with no scaling limits and no SPoF

A short intro to Sharding with pl/proxy is presented.

What else came out of this

As with many simple but novel tools several other, sometimes unexpected, use cases emerged

The ones I'll cover, in no particular order of "importance" are

  • A PostgreSQL-native "microservices architecture"
  • Using pl/proxy and a data-less PostgreSQL proxy instance to expose functionality to CloudSQL that is otherways not available in a managed database, like reading any file system, sending emails or using languages like pl/R or pl/python .
  • Easy and Intuitive way to do map/reduce style data analyses on both multi-core servers and sharded clusters of databases.
  • Autonomous Transactions
  • Sharding BY REFERENCE - it is easy to write apps where all related data is on the same database node even when the identifying column is not in the table.
  • Roll-your-own parallel queries and DML - while PostgreSQL's parallel query capabilities have improved a lot over last 20 years there are still a lot of cases where it does not do what you want. Especially only the driver process can write data.
  • Write "on streaming read-only replica" - RO replicas are by definition read-only. I'll show how you can write a function which a) extracts connect string to master and b) calls a function on aster to write data (which is then immediately replicated back.)
  • Writing progress or debug info from a parallelized query.
  • Running complex in-database jobs
  • Security-at-depth via having only functions on the host the user connects to.
  • the above also helps when one wants to avoid internal structure of tables and source code of functions from ordinary users
  • collecting monitoring info in database at high frequency - up to 2kHz is possible for some metrics.
Back

Join Us For PostgreSQL Conference Europe 2026

October 20–23 2026

Palacio de Congresos, Valencia, Spain