Using REST with the CQRS Pattern to Blend NoSQL & SQL Data

显示全部楼层 · 2014-2-19 11:55:14

REST Easy with SQL/NoSQL Integration and CQRS Pattern implementation
New demands are being put on IT organizations everyday to deliver agile, high-performance, integrated mobile and web applications. In the meantime, the technology landscape is getting complex everyday with the advent of new technologies like REST, NoSQL, Cloud while existing technologies like SOAP and SQL still rule everyday work. Rather than taking religious side of the debate, NoSQL can successfully co-exist with SQL in this ‘polyglot’ of data storage and formats.
However, this integration also adds another layer of complexity both in architecture and implementation.This document offers a guide on how some of the relatively newer technologies like REST can help bridge the gap between SQL and NoSQL with an example of a well known pattern called CQRS. This document is organized as follows:
- Introduction to SQL development process
- NoSQL
- Do I have to choose between SQL and NoSQL?
- CQRS Pattern
- How to implement CQRS pattern using REST services

千问 · 2014-2-19 11:55:14

Introduction to SQL development process
Developers have been using SQL Databases for decades to build and deliver enterprise business applications. The process of creating tables, attributes,and relationships is second nature for most developers.Data architects think in terms oftables and columns and navigate relationships for data.The basic concepts of delivery and transformation takes place at the web server level which means the server developer is reading and ‘binding’ to the tables and mapping attributes to a REST response.
Application development lifecycle meant changes to the database schema first, followed by the bindings, then internal schema mapping, and finally the SOAP or JSON services, and eventually the client code.This all costs the project time and money.It also means that the ‘code’ (pick your language here) and the business logic would also need to be modified to handle the changes to the model.

千问 · 2014-2-19 11:55:14

NoSQL
NoSQLis gaining supporters among many SQL shops for various reasons including:
- Low cost
- Ability to handle unstructured dataa
- Scalability
- Performance
The first thing database folks notice is that there is no schema.These document style storage engines can handle huge volumes of structured, semi-structured, and unstructured data. The very nature of schema-less documents allows change to a document structure without having to go through the formal change management process (or data architect).
The other major difference is that NoSQL (no-schema) also means no joins or relationships.The document itself contains the embedded information by design.So an order entry would contain the customer with all the orders and line items for each order in a single document.
There are many different NoSQL vendors (popular NoSQL databases include MongoDB, Casandra) that are being used for BI and Analytics (read-only) purposes.We are also seeing many customers starting to use NoSQL for auditing, logging, and archival transactions.

千问 · 2014-2-19 11:55:14

Do I have to choose between SQL and NoSQL?
The purpose of this article is to not get into the religious debate about whether to use SQL or NoSQL. Bottom line is both have their place and are suited for certain type of data – SQL for structured data and NoSQL for unstructured data. So why not have the capability to mix and match this data depending on the application.
This can be done by creating a single REST API across both SQL and NoSQL databases. Why a single REST API?The answer is simple – the new agile and mobile world demands this ‘mashup’ of data into a document style JSON response.

千问 · 2014-2-19 11:55:14

CQRS (Command Query Responsibility Segmentation) Pattern
There are many design patterns for delivery of high performance RESTful services but the one that stands out was described in an article written by Martin Fowler, one of the software industry veterans. He described the pattern called CQRS that is more relevant today in a ‘polyglot’ of servers, data, services, and connections.
“We may want to look at the information in a different way to the record store, perhaps collapsing multiple records into one, or forming virtual records by combining information for different places. On the update side we may find validation rules that only allow certain combinations of data to be stored, or may even infer data to be stored that’s different from that we provide.” – Martin Fowler 2011

千问 · 2014-2-19 11:55:14

In this design pattern, the REST API requests (GET) return documents from multiple sources (e.g. mashups). In the update process, the data is subject to business logic derivations, validations, event processing, and database transactions.This data may then be pushed back into the NoSQL using asynchronous events. With the wide-spread adoption of NoSQL databases like MongoDB and schema-less, high capacity data store; most developers are challenged with providing security, business logic, event handling, and integration to other systems.

cqrs1.png (105.22 KB, 下载次数: 11)
下载附件
2014-11-7 16:49 上传

programming language itself is very different from the SQL we all know.

千问 · 2014-2-19 11:55:14

How to implement CQRS pattern using a RESTful Architecture
A REST server should meet certain requirements to support the CQRS pattern. The server should run on-premise or in the cloud and appears to the mobile and web developer as an HTTP endpoint.The server architecture should implement the following:
1) Connections and Mapping necessary for SQL and NoSQL connectivity and API services needed to create and return GET, PUT, POST, and DELETE REST responses
2) Security
3) Business Logic

千问 · 2014-2-19 11:55:14

cqrs2.png (77.98 KB, 下载次数: 4)
下载附件
2014-11-7 16:50 上传

Connections and Mapping
There are two main approaches to creating REST Servers and APIs for SQL and NoSQL databases:
1) Open source frameworks like Apache Tomcat, Spring/Hibernate
2) Commercial framework like Espresso Logic

千问 · 2014-2-19 11:55:14

Open source Frameworks
Using various open source frameworks like Tomcat, Spring/Hibernate, Node.js, JDBC and MongoDB drivers, a REST server can be created, but we would still be left with the following tasks:
1) Creation and mapping of the necessary SQL objects
2) Create a REST server container and configurations
3) Create Jersey/Jackson classes and annotations
4) Create and define REST API for tables, views, and procedures
5) Hand write validation, event and business logic
6) Handle persistence, optimistic locking, transaction paging
7) Adding identity management and security by roles
Now we can start down the same path to connect to MongoDB and write code to connect, select, and return data in JSON and then create the REST calls to merge these two different document styles into a single RESTful endpoint.This is a lot of work for a development team to manage and control and frankly pretty boring and repetitive and is better done by a well designed framework.

千问 · 2014-2-19 11:55:14

Commercial Frameworks
Many commercial frameworks may take care of this complexity without the need to do extensive programming. Here is an example from Espresso Logic and how it handles this complexity with a point and click interface:
1) Running REST server in the cloud or on-premise
2) Connections to external SQL databases
3) Object mapping to tables, views, and procedures
4) Automatic creation of RESTful endpoints from model
5) Reactive business rules and rich event model
6) Integrated role-based security and authentication services.
7) Point-and-click document API creation for SQL and MongoDB endpoints

Using REST with the CQRS Pattern to Blend NoSQL &amp; SQL Data

Using REST with the CQRS Pattern to Blend NoSQL & SQL Data