This site is about a previous instance of this event. For the current event, please see the updated site.

Schedule - PGConf.EU 2017

Success Story on migrating a multi Terabyte analytical Database to Postgres

Date: 2017-10-27
Time: 13:40–14:30
Room: Baltic I
Level: Intermediate

At parcIT we are developing algorithms for the credit industry. To develop and validate their algorithms, our analysts require a large pool of anonymized real world data.

This data pool had been spread on 3 smaller MS-SQL Databases before and is now consolidated to one Postgres DB that has currently grown to 25TB on high performance storage and will be enlarged up to 4 times that size.

The original setup of this Postgres Cluster hit surprisingly many pitfalls and let to a massively instable system with some security concerns. Together they had the potential to lead to a failure of the whole project. It took us several weeks to track the problems down even though all of them were more or less easy to avoid.


Jens Wilke