Comments on: Post-relational thought and the temporal organization of data http://blog.amber.org/2006/08/01/post-relational-thought-and-the-temporal-organization-of-data/ Thoughts of a minor lunatic Wed, 07 Jan 2009 18:17:42 +0000 http://wordpress.org/?v=2.7 hourly 1 By: petrilli http://blog.amber.org/2006/08/01/post-relational-thought-and-the-temporal-organization-of-data/comment-page-1/#comment-20229 petrilli Wed, 02 Aug 2006 00:53:23 +0000 http://blog.amber.org/2006/08/01/post-relational-thought-and-the-temporal-organization-of-data/#comment-20229 "MonetDB":http://monetdb.cwi.nl/ is another example of a vertically-partitioned database that is designed for OLAP-style work. I've played with it some, and for certain types of queries, it's 10x faster, just as you discuss. I also find a lot of the work that Google has done on BigTable to be interesting in this regard as they implement vertical partitioning as well. MonetDB is another example of a vertically-partitioned database that is designed for OLAP-style work. I’ve played with it some, and for certain types of queries, it’s 10x faster, just as you discuss.

I also find a lot of the work that Google has done on BigTable to be interesting in this regard as they implement vertical partitioning as well.

]]> By: Michael Koziarski http://blog.amber.org/2006/08/01/post-relational-thought-and-the-temporal-organization-of-data/comment-page-1/#comment-20212 Michael Koziarski Tue, 01 Aug 2006 22:35:58 +0000 http://blog.amber.org/2006/08/01/post-relational-thought-and-the-temporal-organization-of-data/#comment-20212 In a prior life I built datawarehouses for casinos. The patterns there were: * Huge aggregated reads, never by record. * Daily inserts, perhaps even less frequently. We made use of Sybase's IQ system. While it was a pain in the ass in some circumstances, it was 'fundamentally different' in a number of useful ways. Data was stored by column, not by row. So 'AVG(turnover)' was extremely fast, but select * from whatevers; was much slower. Its optimiser was particularly good at the kind of OLAP queries we'd issue (f.ex http://pastie.caboo.se/6927). On the downsides, INSERTs and UPDATEs were abominable. The custom LOAD TABLE functionality could pull in 100k rows per second, but an insert would often take 4 seconds per row. We had access to SQL, it was optimised for our usage, basically it was great fun. In a prior life I built datawarehouses for casinos. The patterns there were:

  • Huge aggregated reads, never by record.
  • Daily inserts, perhaps even less frequently.

    We made use of Sybase’s IQ system. While it was a pain in the ass in some circumstances, it was ‘fundamentally different’ in a number of useful ways.

    Data was stored by column, not by row. So ‘AVG’ was extremely fast, but select * from whatevers; was much slower.

    Its optimiser was particularly good at the kind of OLAP queries we’d issue (f.ex http://pastie.caboo.se/6927).

    On the downsides, INSERTs and UPDATEs were abominable. The custom LOAD TABLE functionality could pull in 100k rows per second, but an insert would often take 4 seconds per row.

    We had access to SQL, it was optimised for our usage, basically it was great fun.

]]>