Normalization, Standardization and Rescaling

This is copy/paste of an interesting FAQ from faqs.org, I really loved reading the article and thought of reposting the same in a better formatted manner for readability. Courtsey: ftp://ftp.sas.com/pub/neural/FAQ.html First, some definitions Rescaling “Rescaling” a vector means to add or subtract a constant and then multiply or divide by a constant, as you would do to change the units of measurement of the data. For example, to convert a temperature from Celsius to Fahrenheit. [Read More]

Zinda ho tum !

One of the Bollywood movies which I always loved watching has been ZNMD, here is a collection of shayari recited by Farhan Akthar (Imran) in ZNMD. A compiled version from Souncloud. Apne Hone Par Mujhko Yaqeen Aa Gaya (The poem comes after the trio’s deep-sea dive) Pighle neelam sa behta ye sama, neeli neeli si khamoshiyan, na kahin hai zameen na kahin aasmaan, sarsaraati hui tehniyaan pattiyaan, [Read More]

Converting large csv's to nested data structure using apache spark

What is Apache Spark ? Apache Spark brings fast, in-memory data processing to Hadoop. Elegant and expressive development APIs in Scala, Java, and Python allow data workers to efficiently execute streaming, machine learning or SQL workloads for fast iterative access to datasets. Quick start guide Problem Statement / Task To read lot of really big csv’s (~GBs) from Hadoop HDFS, clean, convert them to nested data structure and update it to MongoDB using Apache Spark. [Read More]

Ponmudi - 2

If you happen to be in trivandrum with a bike and you loves to ride, Ponmudi is one place that should go. Sharing some photos taken by Saurab Devanandan during the trip.

The Ponmudi !
Ponmudi

Bullets :)
Our Bikes :)

Me the posing
Sudev

Nabeel the posing
Nabeel

Again Nabeel the posing
Nabeel

Saurab being the one taking pics, his bike the posing
Classic 500

ISRO has some office on top of Ponmudi(Must be fun working here)
ISRO Ponmudi

Data science and unix command line

Note : This article applies only to those who code. I have seen many strugling with MS Excel trying to figure out data in a large csv file, I don’t blame them beacause most people I have met ignore standard unix command line tools just because they cared about commandline tools. When the data is BIG(anything above .5GBs) and if we are trying to figure out say even the coloumn names of a csv file MS Excel will get stuck and we will see a MS Windows Not Responding. [Read More]

Ponmudi

Mountain top and the fog ! Ponmudi top. #hairpins #ride #ponmudi #re A post shared by Sudev Ambadi (@sudev_ambadi) on Nov 8, 2014 at 7:14am PST The peak I'm coming to you peak. #RE #hillstation #ride A post shared by Sudev Ambadi (@sudev_ambadi) on Dec 7, 2014 at 1:05am PST Meemutty waterfals Meemutty waterfalls is on the way to Punmudi top. A nice short off road on the way and about a kilometer walk towards the waterfall. [Read More]

A dictionary in your terminal

Owning to my poor vocabulary I had to look up for meanings every now and then, the following script gets you the meaning for any word from dictionary.com using bash. A bash script dictionary dict() { #Creating a temp folder dir=~/.dict #Check for the existence if not create one [[ -d $dir ]] || mkdir $dir #download respective file from dictionary dot com # -q => do it quietly ie nothing @ screen # -O save it as mean wget -q -O $dir/mean wget http://dictionary. [Read More]

Installing Minix 3 using VirtualBox in Linux

This article will help you in setting up Minix 3 using VirtualBox on a Linux Host for development and enabling ssh connection between Minix and host Linux machine. But why, but why?? Minix is a micro kernel, unlike Linux which is monolithic Minix is a operating system which is Reliable. Being a micro kernel good sepration around the Kernel Space and User Space, Minix Kernel is only 6000 lines of code compared to millions of lines of code in Linux Kernel. [Read More]

Useful bashrc functions

I’m going to share some of my bashrc functions which saves me a lot of time. Killer This function helps you find a process using a keyword and to kill it, you don’t have to use ps aux along with grep and then kill the process by entering the pid instead use this function give it a keyword and it will help you in killing a process. killer() { echo "I'm going to kill these process"; ps -ef | grep $1 | grep -v grep echo "Can I ? [Read More]