Ipython-lectureNotes

' hello world' --> create an array of bits
u'hello world' --> create an string

b= <bit array>
b.decode('utf-8')

range(5)
xrange(10000000000000000) to no create the data in memory
 
Generator:
l=(i*2 for i in range xrange(100000000000000000))

For complex operations is better to use another generator
def doublify(nums):
    for i in nums:
	yield i*2

for i in doublify(xrange(10)):
    print i

g = doublify(xrange(10))

Pass the numbers one by one:
next(g) 

-----------------------------------------------------
Aspect oriented programming:

def print_hello():
    print 'Hello'

print_hello()

def logged(f):
    def logged_f(*args, **kw):
	print 'CALLING{}'.format(f)
	f(*args, **kw)
	print 'DONE'
    return logged_f

**kw = key words arguments
*arg = position arguments

logged_f(1,2,3,4,  a=5, b=6, c=7)


@logged
def print_hello():
    print 'Hello'

@logged
def print_sum(a,b):
	print a+b

print_hello()
print_sum(1, 2)

To know the time to do a task:

def benchmark(f):
	def benchmarked_f(*args, **kw):
		start = time.time()
		f(*args,**kw)
		end = time.time()
		print 'Took: {}'
	return benchmarked_f

@benchmark
@logged
print_hello()
print_sum()

-----------------------
Arguments in functions

def func2(a,b,c):
	print a,b,c

def func(*args, **kwargs):
	l = [args[0]*2, args[1]*3, args[2]*4]
	func2(l)
	func2(*args, **kwargs)

-------------------------
Using the decorator before a function tells the program to use with this function

def logged(text_accessor):
	def logged_deco(f):
	    def logged_f(*args, **kw):
		print '{} {}'.format(text_accessor(), f)
		f(*args, **kw)
		print 'DONE'
	    return logged_f

@logged('START')
def print_hello(name):
	print 'Hello', name

-----------------------------------
Discoproject.org
Open Source Mapper created by NOKIA Research Center

----------------------------------
Hadoop is totally written in Java
Hadoop Distributed File System (HDFS)

Elastic Mapreduce (amazon EMR) is a map reduce claster from Amazon Web Service --> Manage the Hadoop Cluster