Showing posts with label MySQL. Show all posts
Showing posts with label MySQL. Show all posts

Saturday, June 30, 2012

MemSQL - HipHop for SQL

MemSQL is a new database system developed by ex-facebookers to speed up relational databases. As its name implies, it achieves fast performance by keeping data in memory by removing the most common bottleneck "disk" which most applications hit today. Also it supports MySQL client for querying so it doesn't need you to learn anything new and you can easily migrate from MySQL to MemSQL without any change in your application code.

MemSQL takes a lesson learned from HipHop (facebook's tool for converting PHP code into faster C++) and converts SQL to C++. Unlike traditional relational database systems which interpret SQL queries, MemSQL transforms SQL to C++ and compiles the generate code into machine code. This is only one time operation when a query hits first time, and all the future queries which follow the same skeleton (with all the numeric and string parameters stripped off) will bypass code generation and the compiler entirely.

For example, when you execute a query
  SELECT * FROM users WHERE id = 10 and name = "prasad",
it is first scanned through a linear scan parser which strips of all the numeric and string parameters and creates a query skeleton. The resulting query skeleton is just a string and looks like
 SELECT * FROM users WHERE id =@ and name = ^.
This query skeleton is given to code generator which converts it into C++ code that executes the query. The resulting plan is loaded into the database as a shared object and registered in the plan cache.
And all the future queries which match the same skeleton will be directly executed without going through code generation and compilation process.

Having said that MemSQL uses RAM as the primary storage of data, it backs up data to disk with snapshots and transaction logs to not to lose any data on system restart or failure. So your data is durable. These features can be tuned all the way from synchronous durability (every write transaction is recorded on disk before the query completes) to purely in-memory durability (maximum sustained throughput on writes). 

So why wait? I have installed in MemSQL in my 64-bit linux machine today and connected it to using phpmyadmin within 10 minutes. 


Click here to download the developer edition and install it by yourself. 


~ Cheers

Amazon RDS - MySQL - Turn on Slow Log and General Log

Following commands create a new db parameter group for your RDS server and set 
slow_query_log and general_log parameters


You will need to setup RDS tools configured with your keys to execute the below commands


$ rds-create-db-parameter-group slow-log-group -f mysql5.1 -d "This is created to enable slow query logging and a few other db parameters"

$ rds-modify-db-instance pd-rds-server --db-parameter-group-name slow-log-group --apply-immediately

$ rds-modify-db-parameter-group slow-log-group --parameters "name=slow_query_log, value=ON, method=immediate" --parameters "name=long_query_time, value=1, method=immediate" --parameters "name=min_examined_row_limit, value=100, method=immediate" "name=general_log, value=ON, method=immediate"

$ rds-describe-db-parameters slow-log-group

$ rds-reboot-db-instance pd-rds-server