With the release of PostgreSQL 9.3, it let's us do some really cool things with writable foreign tables. BigSQL has just released a Hadoop Foreign Data Wrapper that is writable into HDFS files and Hbase tables. The Hbase integration allows for full SELECT, INSERT, UPDATE and DELETE syntax through PostgreSQL and the HDFS integration allows [...]

By | September 18th, 2013|Categories: Big Data, BigSQL, FDW (Foreign Data Wrapper), Jim PostgreSQL|Tags: , , , , |Comments Off on HadoopFDW

Fluming your PostgreSQL logs

With a default configuration of PostgreSQL, the information in the PostgreSQL logs is generally pretty small with some warnings and the occasional typo in psql. However, once you start tweaking the configuration parameters a bit and you start logging more things like query times, the logs get a lot bigger. Having PostgreSQL generate over 10GB [...]

By | July 18th, 2013|Categories: Jim PostgreSQL|Tags: , , , , |Comments Off on Fluming your PostgreSQL logs

HDFS for PostgreSQL Backups

On several occasions, I've been talking with groups of PostgreSQL users and the question comes up, "If I use PostgreSQL, why would I want to use Hadoop?" There are many answers and the question is usually asked when people don't really understand the details about Hadoop, but let's just focus on a single use case. [...]

By | July 11th, 2013|Categories: Jim PostgreSQL|Tags: , , , |Comments Off on HDFS for PostgreSQL Backups

BigSQL Up and Running

BigSQL Up and Running All Big Data projects are a collection of related projects, each suited to a specific function to implement that Big Data solution. BigSQL can be used as a Big Data Data Warehousing solution. In addition to the archetypical Big Data components (Hadoop Distributed File System and MapReduce), there are additional projects [...]

By | April 30th, 2013|Categories: Big Data, BigSQL, Hadoop, High Availability, OpenSource, PostgreSQL|Tags: , , , , , , , , , |Comments Off on BigSQL Up and Running


BigSQL. From data to information, from information to insight. A state-of-the-art Big Data Warehouse solution that is fast, secure and continuously available. BigSQL will scale from your desktop to the cloud. Run real time OLAP directly from the worlds most secure RDBMS. Get started with BigSQL right now. You can immediately put BigSQL to work [...]

Big Data Implementations

Big Data Implementations. Big Data is conceptually a collection of data and the means to store (a file system) and access it (a runtime). This means that all of the components are swappable including the very core components that were the archetypal definition of Big Data – the file system (HDFS) and the execution environment [...]

By | April 16th, 2013|Categories: Big Data, BigSQL, Fun thoughts, Hadoop|Tags: , , , |Comments Off on Big Data Implementations

The Hadoop Ecosystem

The Hadoop Ecosystem. Big Data is more than just the Apache Hadoop kernel; it is a collection of related projects. Some of them are mandatory (you cannot do Big Data without them) and others are optional (like data loaders and flow processing languages). The mandatory Hadoop projects are HDFS, MapReduce and Common.  Common is a [...]

By | April 16th, 2013|Categories: Big Data, Hadoop, High Availability, OpenSource|Tags: , , , , , |Comments Off on The Hadoop Ecosystem


Hadoop - HDFS and MapReduce. Hadoop came out in 2008 and the primary components of it (HDFS and MapReduce) conceptually look like this. Hadoop, HDFS and MapReduce MapReduce splits work across many machines. Map takes the job, breaks it into many small reproducible pieces and sends them to different machines and runs them [...]

By | April 16th, 2013|Categories: Big Data, Fun thoughts, Hadoop|Tags: , , , , |Comments Off on Hadoop


HDFS, The Hadoop Distributed File System. When you Map a problem you break it down into key value pairs and spread it out across a number of processors – abstract it to its simplest components and give them to lots of workers. HDFS is a file system specifically designed to handle LOTS of key value [...]

By | April 16th, 2013|Categories: Big Data, Fun thoughts, Hadoop, High Availability, OpenSource|Tags: , , , |Comments Off on HDFS

Big Data

Imagine what Big Data can do for you. Big Data is a term, like database, that can mean different but similar things depending on context: Big Data is archetypically the Hadoop Distributed File System (HDFS) and MapReduce. The open source Apache project Hadoop includes HDFS and an implementation of MapReduce. Big Data is a collection [...]

By | April 15th, 2013|Categories: Big Data, BigSQL, Fun thoughts, Hadoop, High Availability, OpenSource|Tags: , , , , , |Comments Off on Big Data