MongoDB restore

To use Mongo DB mongorestore command:

put db dump file in following hierarchy:
dump/<db-name>/<dump-files>

Then go parent of dump folder and type:
mongorestore


It will restore all backup in respective db, depends on db-name where dump files are kept.

Rest Http Status Code, Header (Request-Response)

HTTP codes used in Rest:
 

Http Response Code	HTTP Method	Response Body Contents	Response code Description
200	GET, PUT, DELETE	Resource	No error, operation successful.
201 Created	POST	Resource that was created	Successful creation of a resource.
202 Accepted	POST, PUT, DELETE	N/A	The request was received.
204 No Content	GET, PUT, DELETE	N/A	The request was processed successfully, but no response body is needed.
301 Moved Permanently	GET	XHTML with link	Resource has moved.
303 See Other	GET	XHTML with link	Redirection.
304 Not Modified	conditional GET	N/A	Resource has not been modified.
400 Bad Request	GET, POST, PUT, DELETE	Error Message	Malformed syntax or a bad query.
401 Unauthorized	GET, POST, PUT, DELETE	Error Message	Action requires user authentication.
403 Forbidden	GET, POST, PUT, DELETE	Error Message	Authentication failure or invalid Application ID.
404 Not Found	GET, POST, PUT, DELETE	Error Message	Resource not found.
405 Not Allowed	GET, POST, PUT, DELETE	Error Message	Method not allowed on resource.
406 Not Acceptable	GET	Error Message	Requested representation not available for the resource.
408 Request Timeout	GET, POST	Error Message	Request has timed out.
409 Resource Conflict	PUT, PUT, DELETE	Error Message	State of the resource doesn't permit request.
410 Gone	GET, PUT	Error Message	The URI used to refer to a resource.
411 Length Required	POST, PUT	Error Message	The server needs to know the size of the entity body and it should be specified in the Content Length header.
412 Precondition failed	GET	Error Message	Operation not completed because preconditions were not met.
413 Request Entity Too Large	POST, PUT	Error Message	The representation was too large for the server to handle.
414 Request URI too long	POST, PUT	Error Message	The URI has more than 2k characters.
415 Unsupported Type	POST, PUT	Error Message	Representation not supported for the resource.
416 Requested Range Not Satisfiable	GET	Error Message	Requested range not satisfiable.
500 Server Error	GET, POST, PUT	Error Message	Internal server error.
501 Not Implemented	POST, PUT, DELETE	Error Message	Requested HTTP operation not supported.
502 Bad Gateway	GET, POST, PUT, DELETE	Error Message	Backend service failure (data store failure).
505	GET	Error Message	HTTP version not supported.

Request header in Rest:
 

Request Header	Description
Accept	Request a particular content type. Valid values: application/json, application/xml.
Accept-Encoding	Request content encoding such as gzip.
Accept-Language	Request a particular language. Supported where localized strings are generated, such as Updates event descriptions. Affects sort order. The lang query parameter will override this header.
Authorization	Include appropriate authorization token.
Host	Formats the host portion of embedded links to other social data.
If-Modified-Since	Conditional GET.
If-Unmodified-Since	Conditional GET.

Response header in Rest:

Response Header	Description
Content-Encoding	Included if content has been encoded (gzip).
Content-Language	Language selected.
Content-Length.	Size of response, in bytes, unless content or transfer encoding has been applied.
Content-Type	Type of response.
Last-Modified	The last time the resource data was modified.
Location	URI of a newly created resources.
Transfer-Encoding	The form of encoding used to transfer the entity to the user.

Sources: google,yahoo

Hadoop Terminology

• Hadoop Ecosystem: It consists of Hadoop core project(HDFS, MapReduce) and other open source projects (Hive, Pig, Imapala, Sqoop etc) working on top of Hadoop to help in data analysis
• PIG: High level language to  analyses large data set, internal it uses MR
• Hive: Offers a SQL like language on top of MR
• Imapala: develops as a way to query data in hadoop like SQL with out using MR, with low latency compare to Hive.
• Sqoop: To migrate data from RDBMS to Hadoop
• Flume:  To move data from external resource (logs etc) to Hadoop
• Hbase: real time db on hdfs
• Hue: GUI Frontend to cluster
• Oozie: workflow management
• Mahoot: Machine learning library

Link between SSL, HTTPS, Certificate, Browser, Server

I have tried to draw a link between SSL, HTTPS, Certificate, Browser, Server using simple language in following steps: 

 Process of obtaining Certificate:

•  A web site operator obtains a certificate by applying to a certificate provider (a CA that presents as a commercial retailer of certificates) with a certificate signing request. 
• The certificate provider generate a certificate with public and private key.
• Now web site operator send a request to certificate Authority to sign the request. For security reasons the private key is not part of the request and is not sent to the certificate authority. 
• The certificate provider signs the request, thus producing a public certificate. 
• During web browsing, this public certificate and public key is served to any web browser that connects to the web site and proves to the web browser that the provider believes it has issued a certificate to the owner of the web site.

 SSL flow:

• End-user enter a secure site to connects to
• Browser identify it site is secure, it request SSL certificate from the site.
• SSL certificate contains following information:
• Domain name
• Company name
• City
• State
• Country
• Expiration date
• issuing Authority
• Browser verifies following three items:
• Certificate is not expired
• Certificate has been issued by the Authority that browser trusts
• Certificate has been issued to browser from the site for which it has been generated
• If any of the above condition does not meet then browser shows error for the certificate about the same. ex: This Connection is Untrusted / Certificate Error
• With certificate Browser also receives a Public key
• Based on this initial exchange, browser and the website then initiate the 'SSL handshake'. The SSL handshake involves the generation of shared secrets to establish a uniquely secure connection between yourself and the website. This involves following steps:
• The browser then uses the public key, to encrypt a random symmetric encryption key and sends it to the server with the encrypted URL required as well as other encrypted http data.
• The web server decrypts the symmetric encryption key using its private key and uses the symmetric key to decrypt the URL and http data.
• SSL used asymmetric Public key Infrastructure (PKI). This used two keys: Public keys to encrypt and Private key to decrypt. Anything encrypted with the public key can only be decrypted by the private key and vice-versa.
• In Symmetric Public Key Infrastructure, same key is used for encryption and decryption.

Design pattern Quick Guide

Observer Pattern :  Behavioral pattern

Define a one-to-many dependency between objects so that when one object changes state, all its dependents are notified and updated automatically.

Real Life example:

• RSS feeds.  When I want to get updates from a particular feed, I add it to my feed reader. Any time that the RSS feed has an update, it will appear in my reader automatically.  This is the Observer pattern in action, a publisher/subscriber relationship with one source having many subscribers 
• The whole concept of listeners is based on this pattern.

Adapter Pattern : Structural pattern


Convert the interface of a class into another interface clients expect. Adapter lets classes work together that couldn't otherwise because of incompatible interfaces.

Real Life Example:

• The best example for the adapter pattern is based around AC power adapters.

Downside:
Adapter pattern is just a fix for a badly designed system, which didn't consider all possibilties. While this is a fair point, it is an important part of a pluggable architecture.  It can also add a level of complexity to your code, making debugging more difficult.

The Facade pattern  : Structural pattern

Provide a unified interface to a set of interfaces in a subsystem. Façade defines a higher-level interface that makes the subsystem easier to use.

Real Life Example:

• Concept behind facade is to simplify an interface, service oriented architectures make use of the facade pattern. For example, in web services, one web service might provide access to a number of smaller services that have been hidden from the caller by the facade.

Downside:
 By introducing the Facade into your code, you will be hardwiring subsystems into the Facade. This is fine if the subsystem never changes, but if it does, your Facade could be broken. Therefore, developers working on the subsystem should be made aware of any Facade around their code.

Factory method pattern : Creational pattern

Define an interface for creating an object, but let the subclasses decide which class to instantiate. The Factory method lets a class defer instantiation to subclasses

Real Life Example:

• The idea behind the Factory Method pattern is that it allows for the case where a client doesn't know what concrete classes it will be required to create at run-time, but just wants to get a class that will do the job. The FactoryMethod builds on the concept of a simple Factory, but lets the subclasses decide which implementation of the concrete class to use.  You'll see factories used in logging frameworks, and in a lot of scenarios where the client doesn't need to know about the concrete implementations. It's a good approach to encapsulation.

Abstract Factory Pattern : Creational pattern
Provides an interface for creating families of related or dependent objects without specifying their concrete classes.

Strategy Pattern : Behavioral pattern
Defines a set of encapsulated algorithms that can be swapped to carry out a specific behaviour

Real Life Example:

• Compression algorithm

Visitor Pattern : Behavioral pattern
Allows for one or more operation to be applied to a set of objects at runtime, decoupling the operations from the object structure.

Mongo sharding example

### To Start config server instance (config server port: 27019)
mongod.exe --configsvr --dbpath ../../configdb --port 27019

### To start mongos instance (router) (router port: 27017)
mongos.exe --configdb localhost:27019

###Adding a shard
First start mongoDb instance
mongod.exe --config ../../mongo.cfg
mongod.exe --config ../../mongo1.cfg

Config file content:
##store data here
dbpath=C:\poc\mdb\data

##all output go here
logpath=C:\poc\mdb\log\mongo.log

##log read and write operations
##diaglog=3

###Auth detail admin/admin
auth=false

##Port binding
port=27020

###Open a new instance and add above mongodb instance in cluster:
mongo.exe --host localhost --port 27017 
once mongos console connected:
 sh.addShard("localhost:27020")
 sh.addShard("localhost:27021") 

sh.enableSharding("ankur")
use config
db.databases.find()

###Insert bulk data

for (var i = 1; i <= 1000; i++) db.test_collection.insert( { x : i } )

###To know the shard status

sh.status()

###To shard a collection

 sh.shardCollection("ankur.records",{"zip":1, "name":1})

###Data insertion

{
db.records.insert( { zip : 60111 , name: "ankur"})
db.records.insert( { zip : 60111 , name: "goel"})
db.records.insert( { zip : 50111 , name: "ankur"})
db.records.insert( { zip : 50111 , name: "goel"})
db.records.insert( { zip : 70111 , name: "dr"})
} 

###To know chunk distribution
db.records.getShardDistribution()


Source: http://docs.mongodb.org/manual/reference/method/#sharding

A day with Maven

POM: Project Object Model

Maven has three life cycle:

• Clean
• default (build)
• Site

Clean Life cycle has following phases:

• Pre-Clean
• Clean
• Post-Clean

Build(or Default) life cycle has following phase(primarily used) sequence. (Build Life cycle has 23 phases):

• prepare-resources
• compile
• package
• Install

Site life-cycle has following phases:

• pre-site
• site
• post-site
• site-deploy

Syntax for life-cycle and phase:
mvn <life-cycle> : <phase>
ex: mvn clean:clean

Syntax for activating profile:
mvn <life-cycle>/<phase>   -P<profile>
ex: mvn clean -P phix


When a phase is called via Maven command, for example mvn compile, only phases upto and including that phase will execute.

In Maven terminology, a repository is a place i.e. directory where all the project jars, library jar, plugins or any other project specific artifacts are stored and can be used by Maven easily.

Maven repository are of three types

•     local
•     central (by default: http://repo1.maven.org/maven2/) can be browsed: http://search.maven.org/#browse
•     remote (if not available in local and central repository, then if configured searched)

Maven is actually a plugin execution framework where every task is actually done by plugins.

A plugin generally provides a set of goals and which can be executed using following syntax:
mvn <plugin-name>:<goal-name>

Maven provided two types of plugin:

• Build plugins (They execute during the build and should be configured in the <build/> element of pom.xml)
• Reporting plugins (They execute during the site generation and they should be configured in the <reporting/> element of the pom.xml)

Dependency Management:
The dependency management section is a mechanism for centralizing dependency information. When you have a set of projects that inherits a common parent it's possible to put all information about the dependency in the common POM and have simpler references to the artifacts in the child POMs.

For parent:

<dependencyManagement>
   <dependencies>
     <dependency>
       <groupId>test</groupId>
       <artifactId>a</artifactId>
       <version>1.2</version>
     </dependency>

   </dependencies> 

</dependencyManagement>

For child: If below code is not copied in child then no dependency will be included in child project from parent. Need to define all required dependency in child project.

  <dependencies>
    <dependency>
      <groupId>group-a</groupId>
      <artifactId>artifact-a</artifactId>
    </dependency>

 </dependencies>