Basic Python Training at Arcesium - Day 3

Oct 25-31, 2018 Vikrant Patil

These notes are available online at http://notes.pipal.in/2018/arcesium-basic-oct/day3.html © Pipal Academy LLP

Day 1 | Day 2 | Day 3 | Day 4 | Day 5

We will be using python 3 (>= 3.0) from anaconda for this training. You can download it from

https://www.anaconda.com/download/

looping ..continued

problem

  • write a function double which doubles every item from a list and returns new list
  • Write a function which squares every item from a list and returns it.
  • Write a function which finds product of all elements from a list.
  • make use of product to write a function to find factorial of given number
In [1]:
integers = list(range(10))
In [2]:
integers
Out[2]:
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
In [3]:
for i in integers:
    print(i*i, end=",")

0,1,4,9,16,25,36,49,64,81,
In [4]:
def double(numbers):
    d = []
    for n in numbers:
        d.append(2*n)
    return d
In [8]:
d = double([23,324,1,4,65])
d
Out[8]:
[46, 648, 2, 8, 130]
In [6]:
double(["hello", "some", "strings", "to", "have", "fun"])
Out[6]:
['hellohello', 'somesome', 'stringsstrings', 'toto', 'havehave', 'funfun']
In [9]:
len(d)
Out[9]:
5
In [10]:
d[4]
Out[10]:
130
In [11]:
d[6]

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-11-a1943ff16815> in <module>()
----> 1 d[6]

IndexError: list index out of range
In [12]:
empty = []
In [13]:
empty[0] = 0

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-13-d37e925836a7> in <module>()
----> 1 empty[0] = 0

IndexError: list assignment index out of range
In [14]:
empty.append(0)
In [15]:
range(10)
Out[15]:
range(0, 10)
In [16]:
list(range(10))
Out[16]:
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
In [17]:
list(range(1,10))
Out[17]:
[1, 2, 3, 4, 5, 6, 7, 8, 9]
In [19]:
list(range(1, 20, 3))
Out[19]:
[1, 4, 7, 10, 13, 16, 19]
In [20]:
def square(numbers):
    s = []
    for n in numbers:
        s.append(n*n)
    return s
In [21]:
def product(numbers):
    p = 1
    for n in numbers:
        p = p*n
    return p
In [22]:
product([3,1,2,3])
Out[22]:
18
In [23]:
product([1,2,3,4])
Out[23]:
24
In [24]:
product([1,2,3,4,5])
Out[24]:
120
In [25]:
range(1,5)
Out[25]:
range(1, 5)
In [26]:
def factorial(n):
    return product(range(1,n+1))
In [27]:
factorial(7)
Out[27]:
5040
In [28]:
factorial(4)
Out[28]:
24
In [29]:
def even(n):
    return n%2==0

def odd(n):
    return not even(n)


def evens(numbers):
    e = []
    for n in numbers:
        if even(n):
            e.append(n)
    return e

def odds(numbers):
    o = []
    for n in numbers:
        if odd(n):
            o.append(n)
    return o
In [30]:
[n*n for n in range(5)]
Out[30]:
[0, 1, 4, 9, 16]
In [31]:
[2*d for d in range(7)]
Out[31]:
[0, 2, 4, 6, 8, 10, 12]
In [32]:
def doublec(numbers):
    return [n*n for n in numbers]
In [33]:
doublec(range(5))
Out[33]:
[0, 1, 4, 9, 16]
In [34]:
5%2
Out[34]:
1
In [35]:
5%3
Out[35]:
2
In [36]:
5/3
Out[36]:
1.6666666666666667
In [37]:
5//3
Out[37]:
1
In [38]:
def percent(perc, value):
    return perc/100.0*value
In [39]:
percent(10, 200)
Out[39]:
20.0
In [40]:
def squareodds(numbers):
    return [n*n for n in numbers if odd(n)]
In [41]:
squareodds(range(20))
Out[41]:
[1, 9, 25, 49, 81, 121, 169, 225, 289, 361]
In [42]:
odds(range(20))
Out[42]:
[1, 3, 5, 7, 9, 11, 13, 15, 17, 19]

problem

  • write a function double which doubles every item from a list and returns new list
  • Write a function which finds 10% of every item from a list and returns it.
  • find sum of all multipliers if 7 or 11 less than 1000
In [43]:
if even(5) or 5==5:
    print("even!")

even!
In [44]:
"hel" in "hello" and "hello".endswith("lo")
Out[44]:
True
In [45]:
def double(nums):
    return [2*x for x in nums]

def perc10(values):
    return [percent(10, v) for v in values]

sum([n for n in range(1000) if n%7==0 or n%11==0])
Out[45]:
110110
In [46]:
tables = [[i*j for i in range(1,11)]  for j in range(1,6) ]
In [47]:
tables
Out[47]:
[[1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
 [2, 4, 6, 8, 10, 12, 14, 16, 18, 20],
 [3, 6, 9, 12, 15, 18, 21, 24, 27, 30],
 [4, 8, 12, 16, 20, 24, 28, 32, 36, 40],
 [5, 10, 15, 20, 25, 30, 35, 40, 45, 50]]
In [48]:
tables[1]
Out[48]:
[2, 4, 6, 8, 10, 12, 14, 16, 18, 20]
In [49]:
tables[0]
Out[49]:
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
In [50]:
tables[3]
Out[50]:
[4, 8, 12, 16, 20, 24, 28, 32, 36, 40]
In [51]:
tables[4]
Out[51]:
[5, 10, 15, 20, 25, 30, 35, 40, 45, 50]
In [52]:
tables[0]  # gives me 0th row
Out[52]:
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
In [53]:
tables[-1] # last row
Out[53]:
[5, 10, 15, 20, 25, 30, 35, 40, 45, 50]
In [54]:
tables
Out[54]:
[[1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
 [2, 4, 6, 8, 10, 12, 14, 16, 18, 20],
 [3, 6, 9, 12, 15, 18, 21, 24, 27, 30],
 [4, 8, 12, 16, 20, 24, 28, 32, 36, 40],
 [5, 10, 15, 20, 25, 30, 35, 40, 45, 50]]
In [57]:
row0 = tables[0]
In [58]:
row0[0]
Out[58]:
1
In [59]:
row1 = tables[1]
In [60]:
row1[0]
Out[60]:
2
In [62]:
def column(data, colnum):
    return [row[colnum] for row in data]
In [63]:
column(tables, 0)
Out[63]:
[1, 2, 3, 4, 5]
In [64]:
column(tables, 1)
Out[64]:
[2, 4, 6, 8, 10]
In [65]:
def column_(data, colnum):
    return [data[i][colnum] for i in range(len(data))]
In [68]:
def transpose(data):
    colcount = len(data[0])
    return [column(data, i) for i in range(colcount)]
In [69]:
transpose(tables)
Out[69]:
[[1, 2, 3, 4, 5],
 [2, 4, 6, 8, 10],
 [3, 6, 9, 12, 15],
 [4, 8, 12, 16, 20],
 [5, 10, 15, 20, 25],
 [6, 12, 18, 24, 30],
 [7, 14, 21, 28, 35],
 [8, 16, 24, 32, 40],
 [9, 18, 27, 36, 45],
 [10, 20, 30, 40, 50]]

Files

In [70]:
%%file data.csv
1,2,3,4,5
11,12,13,14,15
21,22,23,24,25
31,32,33,34,35,
41,42,43,44,45

Writing data.csv
In [72]:
with open("data.csv") as f:
    for line in f:
        print(line, end="")

1,2,3,4,5
11,12,13,14,15
21,22,23,24,25
31,32,33,34,35,
41,42,43,44,45
In [73]:
f = open("data.csv")
for line in f:
    print(line, end="")
f.close()

1,2,3,4,5
11,12,13,14,15
21,22,23,24,25
31,32,33,34,35,
41,42,43,44,45
In [74]:
!cat data.csv

1,2,3,4,5
11,12,13,14,15
21,22,23,24,25
31,32,33,34,35,
41,42,43,44,45
In [75]:
!head data.csv

1,2,3,4,5
11,12,13,14,15
21,22,23,24,25
31,32,33,34,35,
41,42,43,44,45
In [76]:
f = open("data.csv")
print(f.read())

1,2,3,4,5
11,12,13,14,15
21,22,23,24,25
31,32,33,34,35,
41,42,43,44,45
In [77]:
f.close()
In [78]:
!head paytm.csv
In [80]:
f = open("data.csv")
print(f.readline(), end="")

1,2,3,4,5
In [81]:
f.readline()
Out[81]:
'11,12,13,14,15\n'
In [82]:
f.readline()
Out[82]:
'21,22,23,24,25\n'
In [83]:
f.readline()
Out[83]:
'31,32,33,34,35,\n'
In [84]:
f.readline()
Out[84]:
'41,42,43,44,45'
In [85]:
f.readline()
Out[85]:
''
In [86]:
f.close()
In [87]:
f = open("data.csv")
f.readlines()
Out[87]:
['1,2,3,4,5\n',
 '11,12,13,14,15\n',
 '21,22,23,24,25\n',
 '31,32,33,34,35,\n',
 '41,42,43,44,45']
In [88]:
f.read()
Out[88]:
''
In [89]:
with open("numbers.txt", "w") as nums:
    nums.write("one\n")
    nums.write("two\n")
    nums.write("three\n")
In [90]:
!cat numbers.txt

one
two
three
In [93]:
%%file cat.py
import sys

def cat(file):
    with open(file) as f:
        print(f.read())
        
if __name__ == "__main__":
    cat(sys.argv[1])

Overwriting cat.py
In [94]:
!python cat.py numbers.txt

one
two
three

problem

  • Write a python module head.py which shows first n lines of file 
    python head.py 2 data.csv
1,2,3,4,5
11,12,13,14,15
In [95]:
%%file head.py
import sys

def head(filename, n):
    with open(filename) as f:
        for i in range(n):
            print(f.readline(), end="")
            

if __name__ == "__main__":
    head(sys.argv[1], int(sys.argv[2]))

Writing head.py
In [97]:
!python head.py day1.html 3

<!DOCTYPE html>
<html>
<head><meta charset="utf-8" />

[1,2,3,4,5] 1,2,3,4,5

Write data in csv format

In [121]:
def writecsv(data, filename):
    with open(filename, "w") as f:
        for row in data:
            strrow = [str(item) for item in row] # ["1","2","3","4","5"]
            line = ",".join(strrow) #  "1,2,3,4,5"
            f.write(line)
            f.write("\n")
In [122]:
writecsv(tables, "tables.txt")
In [123]:
!python cat.py tables.txt

1,2,3,4,5,6,7,8,9,10
2,4,6,8,10,12,14,16,18,20
3,6,9,12,15,18,21,24,27,30
4,8,12,16,20,24,28,32,36,40
5,10,15,20,25,30,35,40,45,50
In [117]:
for row in tables:
    print(str(row))

[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
[2, 4, 6, 8, 10, 12, 14, 16, 18, 20]
[3, 6, 9, 12, 15, 18, 21, 24, 27, 30]
[4, 8, 12, 16, 20, 24, 28, 32, 36, 40]
[5, 10, 15, 20, 25, 30, 35, 40, 45, 50]
In [116]:
for row in tables:
    print([str(item) for item in row])

['1', '2', '3', '4', '5', '6', '7', '8', '9', '10']
['2', '4', '6', '8', '10', '12', '14', '16', '18', '20']
['3', '6', '9', '12', '15', '18', '21', '24', '27', '30']
['4', '8', '12', '16', '20', '24', '28', '32', '36', '40']
['5', '10', '15', '20', '25', '30', '35', '40', '45', '50']
In [118]:
def csvparse(filename):
    with open(filename) as f:
        data= []
        for line in f:
            data.append(line.strip().split(","))
        return data
In [124]:
csvparse("tables.txt")
Out[124]:
[['1', '2', '3', '4', '5', '6', '7', '8', '9', '10'],
 ['2', '4', '6', '8', '10', '12', '14', '16', '18', '20'],
 ['3', '6', '9', '12', '15', '18', '21', '24', '27', '30'],
 ['4', '8', '12', '16', '20', '24', '28', '32', '36', '40'],
 ['5', '10', '15', '20', '25', '30', '35', '40', '45', '50']]
In [127]:
def csvparse(filename):
    with open(filename) as f:
        return [line.strip().split(",") for line in f]
In [128]:
csvparse("tables.txt")
Out[128]:
[['1', '2', '3', '4', '5', '6', '7', '8', '9', '10'],
 ['2', '4', '6', '8', '10', '12', '14', '16', '18', '20'],
 ['3', '6', '9', '12', '15', '18', '21', '24', '27', '30'],
 ['4', '8', '12', '16', '20', '24', '28', '32', '36', '40'],
 ['5', '10', '15', '20', '25', '30', '35', '40', '45', '50']]
In [129]:
def csvparse(filename):
    def makeints(row):
        return [int(item) for item in row]
    
    with open(filename) as f:
        return [makeints(line.strip().split(",")) for line in f]
In [131]:
t = csvparse("tables.txt")
In [132]:
t
Out[132]:
[[1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
 [2, 4, 6, 8, 10, 12, 14, 16, 18, 20],
 [3, 6, 9, 12, 15, 18, 21, 24, 27, 30],
 [4, 8, 12, 16, 20, 24, 28, 32, 36, 40],
 [5, 10, 15, 20, 25, 30, 35, 40, 45, 50]]
In [134]:
type(t[0][0])
Out[134]:
int
In [136]:
%%file tables.csv
c1,c2,c3,c4,c5,c6,c7,c8,c9,c10
1,2,3,4,5,6,7,8,9,10
2,4,6,8,10,12,14,16,18,20
3,6,9,12,15  ,18,,24,27,30
4,8,12,16,20,24,28,32,36,40
5,10,15,20,25,30,35,40,,50

Overwriting tables.csv
In [137]:
csvparse("tables.csv")

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-137-aa51ccf0d490> in <module>()
----> 1 csvparse("tables.csv")

<ipython-input-129-884d6b9d7966> in csvparse(filename)
      4 
      5     with open(filename) as f:
----> 6         return [makeints(line.strip().split(",")) for line in f]

<ipython-input-129-884d6b9d7966> in <listcomp>(.0)
      4 
      5     with open(filename) as f:
----> 6         return [makeints(line.strip().split(",")) for line in f]

<ipython-input-129-884d6b9d7966> in makeints(row)
      1 def csvparse(filename):
      2     def makeints(row):
----> 3         return [int(item) for item in row]
      4 
      5     with open(filename) as f:

<ipython-input-129-884d6b9d7966> in <listcomp>(.0)
      1 def csvparse(filename):
      2     def makeints(row):
----> 3         return [int(item) for item in row]
      4 
      5     with open(filename) as f:

ValueError: invalid literal for int() with base 10: 'c1'
In [138]:
def csvparse(filename):
    def myint(ns):
        try:
            return int(ns)
        except Exception as e:
            print(e)
            return 0
        
    def makeints(row):
        return [myint(item) for item in row]
    
    with open(filename) as f:
        headers = f.readline().strip().split(",")
        return [headers] +  [makeints(line.strip().split(",")) for line in f]
In [140]:
t = csvparse("tables.csv")

invalid literal for int() with base 10: ''
invalid literal for int() with base 10: ''
In [141]:
t[0]
Out[141]:
['c1', 'c2', 'c3', 'c4', 'c5', 'c6', 'c7', 'c8', 'c9', 'c10']
In [142]:
t[1:]
Out[142]:
[[1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
 [2, 4, 6, 8, 10, 12, 14, 16, 18, 20],
 [3, 6, 9, 12, 15, 18, 0, 24, 27, 30],
 [4, 8, 12, 16, 20, 24, 28, 32, 36, 40],
 [5, 10, 15, 20, 25, 30, 35, 40, 0, 50]]
In [143]:
sum(column(t[1:], 0))
Out[143]:
15
In [144]:
import numpy as np
In [145]:
np.irr(t[1:][0])
Out[145]:
nan
In [146]:
np.mean(t[1:][0])
Out[146]:
5.5
In [147]:
np.mean(t[1:])
Out[147]:
15.18
In [148]:
[np.mean(row) for row in t[1:]]
Out[148]:
[5.5, 11.0, 14.4, 22.0, 23.0]
In [149]:
int(" ")

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-149-75af607fcdd5> in <module>()
----> 1 int(" ")

ValueError: invalid literal for int() with base 10: ' '
In [150]:
int("-")

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-150-bea2acd6979b> in <module>()
----> 1 int("-")

ValueError: invalid literal for int() with base 10: '-'
In [151]:
try:
    print(int("-"))
except Exception as e:
    print(e)
    print(0)

invalid literal for int() with base 10: '-'
0
In [153]:
url = " http://notes.pipal.in/2018/arcesium-basic-oct/paytm.csv"
import requests
def download(url, filename):
    response = requests.get(url)
    with open(filename, "w") as f:
        f.write(response.text)
In [154]:
download(url, "paytm1.csv")
In [156]:
!python head.py paytm1.csv 5

"Date","Activity","Source/Destination","Wallet Txn ID","Comment","Debit","Credit","Transaction Breakup"
"03/01/2018 12:13:20","Bonus Added","Paytm Order #CST_BUS_TICKETS-NEW-BONUS-12791","17290661578","","","39",""
"03/01/2018 12:10:04","Restored to Paytm Cash against failed order","Paytm BUS Order #4371207130","17290624804","","","403",""
"03/01/2018 12:09:48","Restored to Paytm Cash against failed order","Paytm BUS Order #4371207130","17290621887","","","403",""
"02/01/2018 19:13:00","Added to Paytm Cash","Paytm Order #4378724688","17283050777","","","3000",""
In [159]:
def paytm_parse(file):
    with open(file) as f:
        data = []
        for line in f:
            row = line.strip().split(",")
            row_ = [item.replace('"',"") for item in row]
            data.append(row_)
        return data
In [161]:
paytm = paytm_parse("paytm.csv")
In [162]:
paytm[0]
Out[162]:
['Date',
 'Activity',
 'Source/Destination',
 'Wallet Txn ID',
 'Comment',
 'Debit',
 'Credit',
 'Transaction Breakup']
In [165]:
date= column(paytm[1:], 0)
activity = column(paytm[1:], 1)
src_dest = column(paytm[1:], 2)
txid = column(paytm[1:], 3)
comment = column(paytm[1:], 4)
debit = column(paytm[1:], 5)
credit = column(paytm[1:], 6)
breakup = column(paytm[1:], 7)
In [166]:
date[:5]
Out[166]:
['03/01/2018 12:13:20',
 '03/01/2018 12:10:04',
 '03/01/2018 12:09:48',
 '02/01/2018 19:13:00',
 '01/01/2018 16:38:04']
In [167]:
debit[:5]
Out[167]:
['', '', '', '', '99']
In [168]:
def myfloat(sf):
    try:
        return float(sf)
    except ValueError as v:
        return 0

debit_ = [myfloat(item) for item in debit]
credit_ = [myfloat(item) for item in credit]
In [169]:
debit_[:5]
Out[169]:
[0, 0, 0, 0, 99.0]
In [170]:
sum(debit_)
Out[170]:
48494.58
In [171]:
sum(credit_)
Out[171]:
53120.3
In [175]:
activity[:20]
Out[175]:
['Bonus Added',
 'Restored to Paytm Cash against failed order',
 'Restored to Paytm Cash against failed order',
 'Added to Paytm Cash',
 'Paid for Order',
 'Bonus Added',
 'Bonus Added',
 'Paid for Order',
 'Restored to Paytm Cash against failed order',
 'Paid for Order',
 'Paid for Order',
 'Paid for Order',
 'Paid for Order',
 'Paid for Order',
 'Added to Paytm Cash',
 'Paid for Order',
 'Paid for Order',
 'Paid for Order',
 'Paid for Order',
 'Bonus Added']
In [191]:
def checkifexists(pattern, col):
    for item in col:
        if pattern in item.lower():
            return True
    return False
In [192]:
checkifexists("uber", activity)
Out[192]:
False
In [193]:
checkifexists("uber", comment)
Out[193]:
False
In [194]:
checkifexists("uber", src_dest)
Out[194]:
True
In [195]:
checkifexists("uber", txid)
Out[195]:
False
In [196]:
"uber" in  "UBER Order #2df540e2a73".lower()
Out[196]:
True
In [197]:
src_dest[:50]
Out[197]:
['Paytm Order #CST_BUS_TICKETS-NEW-BONUS-12791',
 'Paytm BUS Order #4371207130',
 'Paytm BUS Order #4371207130',
 'Paytm Order #4378724688',
 'Paytm Order #4372700189',
 'Paytm Order #CASH-667743869',
 'Paytm Order #CASH-667743868',
 'Paytm BUS Order #4371207130',
 'Paytm Order #4365510241',
 'Paytm Order #4365510218',
 'Paytm Order #4365510241',
 'Paytm Order #4331644394',
 'UBER Order #fc814aee07043f07bdfa55c759e35a0',
 'UBER Order #2df540e2a73c3d55a4c035ded4c47a0',
 'Paytm Order #4292715590',
 'redbus Order #39662092',
 'UBER Order #6b3d39482a073d84bae3cebb139f9a0',
 'UBER Order #7aab90717ce23845b3921892a0253a0',
 'UBER Order #7a50e05172c03a6bbbe5b1ba73ad1a0',
 'Paytm Order #17068993541_cashback',
 'Order #SM_1CE10B1AA617BDA8',
 'UBER Order #6f607613d8fa3ee99dc2d2c5d9deda0',
 'UBER Order #a55e9f11a2413befa3a4ea081c032a0',
 'Paytm Order #17052777528_cashback',
 'Order #QR1513323783283',
 'Falcon 7 Order #QR4ADB6670F0DA3EAE',
 'UBER Order #1709f99f6523325eb9424259a076aa0',
 'UBER Order #4e56591df2233c2e9559fba80a555a0',
 'Paytm Order #17035256242_cashback',
 'Order #QR1513237003491',
 'UBER Order #30b17957509a3d92a33cb8f239aafa0',
 'UBER Order #b7a55bfef6b03bc69cf859e1a0414a0',
 'Paytm Order #17018952399_cashback',
 'Order #QR1513151296749',
 'UBER Order #76c8cdf4552e338eb672c5f6c3fb9a0',
 'UBER Order #8fb12fa743a03c0da560594902797a0',
 'UBER Order #ab266f4f66e13eb683d93aca096a6a0',
 'Paytm Order #4267654627',
 'UBER Order #22f0f0a58da63738b28a9e66ff4e3a0',
 'UBER Order #c02fdd41963e32db9a5f5dafdcaa4a0',
 'UBER Order #ee3c7194555a33628267a78316dada0',
 'Paytm Order #CASH-646367492',
 'Paytm BUS Order #4236033718',
 'Paytm Order #CASH-646364164',
 'Paytm Order #CASH-646364163',
 'Paytm BUS Order #4236002969',
 'Paytm Order #4194496739',
 'Paytm Order #4181287087',
 'Paytm Order #4181270403',
 'UBER Order #796cffcb45903114abae9a8f0f6b1a0']
In [201]:
def sumif(pattern, lookupdata, data):
    l = len(lookupdata)
    indices = [ i for i in range(l) if pattern in lookupdata[i].lower()]
    return sum([data[j] for j in indices])
In [202]:
sumif("x", ["x","x","X","y","y","z"], [1,1,1,2,3,4])
Out[202]:
3
In [203]:
sumif("uber", src_dest, debit_)
Out[203]:
15873.990000000005
In [ ]: