Python Training at VMWare Bangalore - Day¶

Sep 18-20 2017 Vikrant Patil

These notes are available online at http://notes.pipal.in/2017/vmware-python

Working with files¶

%%file three.txt
one
two
three

Writing three.txt

fhandle = open("three.txt")

fhandle.read() # will read complete contents of file in single statement

'one\ntwo\nthree'

fhandle.read()

''

fhandle.close()

!python -c "import this" > data.txt

filehandle = open("data.txt")

filehandle.readline()

'The Zen of Python, by Tim Peters\n'

lines = filehandle.readlines()

print(lines)

['\n', 'Beautiful is better than ugly.\n', 'Explicit is better than implicit.\n', 'Simple is better than complex.\n', 'Complex is better than complicated.\n', 'Flat is better than nested.\n', 'Sparse is better than dense.\n', 'Readability counts.\n', "Special cases aren't special enough to break the rules.\n", 'Although practicality beats purity.\n', 'Errors should never pass silently.\n', 'Unless explicitly silenced.\n', 'In the face of ambiguity, refuse the temptation to guess.\n', 'There should be one-- and preferably only one --obvious way to do it.\n', "Although that way may not be obvious at first unless you're Dutch.\n", 'Now is better than never.\n', 'Although never is often better than *right* now.\n', "If the implementation is hard to explain, it's a bad idea.\n", 'If the implementation is easy to explain, it may be a good idea.\n', "Namespaces are one honking great idea -- let's do more of those!\n"]

lines

['\n',
 'Beautiful is better than ugly.\n',
 'Explicit is better than implicit.\n',
 'Simple is better than complex.\n',
 'Complex is better than complicated.\n',
 'Flat is better than nested.\n',
 'Sparse is better than dense.\n',
 'Readability counts.\n',
 "Special cases aren't special enough to break the rules.\n",
 'Although practicality beats purity.\n',
 'Errors should never pass silently.\n',
 'Unless explicitly silenced.\n',
 'In the face of ambiguity, refuse the temptation to guess.\n',
 'There should be one-- and preferably only one --obvious way to do it.\n',
 "Although that way may not be obvious at first unless you're Dutch.\n",
 'Now is better than never.\n',
 'Although never is often better than *right* now.\n',
 "If the implementation is hard to explain, it's a bad idea.\n",
 'If the implementation is easy to explain, it may be a good idea.\n',
 "Namespaces are one honking great idea -- let's do more of those!\n"]

filehandle = open("data.txt")

for line in filehandle.readlines():
    words = line.strip().split()
    print(len(words))

7
0
5
5
5
5
5
5
2
9
4
5
3
10
13
12
5
8
11
13
12

Do it yourself¶

Write a program cat.py equivalent to cat command in
```
python cat.py three.txt
one
two
three
```

Write a program head.py equivalent to unix command head. it should take first commandline argument as number of lines and second argument as filename

python head.py 5 data.txt
The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.

%%file cat.py
import sys

def print_file(filename):
    f = open(filename)
    for line in f.readlines():
        print(line.strip())
    
if __name__ == "__main__":
    print_file(sys.argv[1])

Writing cat.py

!python cat.py three.txt

one
two
three

%%file head.py
import sys

def head(filename, n):
    f = open(filename)
    
    for line in f.readlines()[:n]:
        print(line.strip())

if __name__ == "__main__":
    head(sys.argv[1], int(sys.argv[2]))

Overwriting head.py

!python head.py data.txt 5

The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.

%%file wc.py
"""
module wc implements unix equivalnet of wc cammand
"""
import sys

def line_count(f):
    lines = open(f).readlines()
    return len(lines)

def word_count(f):
    words = open(f).read().split()
    return len(words)
    
def char_count(f):
    return len(open(f).read())


if __name__ == "__main__":
    f = sys.argv[1]
    print(line_count(f), word_count(f), char_count(f))

Overwriting wc.py

!python wc.py data.txt

21 144 857

import os

files = [f for f in os.listdir(".") if f.endswith(".txt")]
files

['three.txt', 'yes.txt', 'data.txt', 'other.txt']

import wc
max(files, key=wc.line_count)

'yes.txt'

max(files, key=wc.word_count)

'other.txt'

max(files, key=wc.char_count)

'other.txt'

Writing files¶

f = open("primes.txt", "w")

f.write("two\n")
f.write("five\n")
f.write("three\n")
f.close()

!python cat.py primes.txt

two
five
three

f = open("primes.txt", "a")
f.write("seven\n")
f.write("eleven\n")
f.close()

!python cat.py primes.txt

two
five
three
seven
eleven

Similarly we can read and write binary files with following mode

rb => read in binary mode
wb => write in binary mode
ab => append in binary mode

open("primes.txt", "r").read() # read in text mode

'two\nfive\nthree\nseven\neleven\n'

open("primes.txt", "rb").read()

b'two\nfive\nthree\nseven\neleven\n'

f = open("binarydata.bin", "wb")
f.write(b'x025x082')
f.close()

open("binarydata.bin", "rb").read()

b'x025x082'

f = open("binarydata.bin", "ab")
f.write(b'hello')
f.close()

open("binarydata.bin", "rb").read()

b'x025x082hello'

with statement¶

with open("primes.txt", "a") as f:
    f.write("thirteen")

!python cat.py primes.txt

two
five
three
seven
eleven
thirteen

with open("regional.txt", "w", encoding='utf-8') as regional:
    regional.write("\u0c05\u0c06")

!python cat.py regional.txt

అఆ

open("regional.txt", encoding='utf-8').read()

'అఆ'

open("regional.txt", 'rb').read()

b'\xe0\xb0\x85\xe0\xb0\x86'

Working with dictionaries¶

author = {'name':"lewis carrol",
        "books": ["alice in wonderland", "looking through the glass"],
         "language" : "English"}

author['name'] = "lewis"

author

{'books': ['alice in wonderland', 'looking through the glass'],
 'language': 'English',
 'name': 'lewis'}

del author['language']

author

{'books': ['alice in wonderland', 'looking through the glass'],
 'name': 'lewis'}

'name' in author

True

author['name']

'lewis'

author.get('name')

'lewis'

author['language']

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-81-808b3817d72f> in <module>()
----> 1 author['language']

KeyError: 'language'

author.get("language", "English")

'English'

author

{'books': ['alice in wonderland', 'looking through the glass'],
 'name': 'lewis'}

author.get("language")

author.get("language") == None

True

author.get("books")

['alice in wonderland', 'looking through the glass']

del author['books']

author

{'name': 'lewis'}

author.get("books", [])

[]

Iterating over dictionary¶

d = {"one":1, "two":2, "three":3}

for key in d.keys():
    print(key, d[key])

one 1
three 3
two 2

for values in d.values():
    print(values)

1
3
2

for key, value in d.items():
    print(key, value)

one 1
three 3
two 2

What if we iterate over dictionary directly?

for item in d:
    print(item)

one
three
two

numbers = [("one", 1), ("two", 2), ("three", 3)]

dict(numbers)

{'one': 1, 'three': 3, 'two': 2}

items = ("pen", "pencil", "colorbox")
prices = (25, 10, 50)
cart = dict(zip(items, prices))

cart

{'colorbox': 50, 'pen': 25, 'pencil': 10}

for item, price in cart.items():
    print(item.rjust(8), price)
print("-"*12)
print("Total".rjust(8), sum(cart.values()))

  pencil 10
colorbox 50
     pen 25
------------
   Total 85

can you write a function unzip which retuns two separate lists of keys and values

def unzip(d):
    return list(d.keys()), list(d.values())

unzip(cart)

(['pencil', 'colorbox', 'pen'], [10, 50, 25])

%%file words.txt
five
five four
five four three
five four three two
five four three two one
six seven eight nine
six seven eight
six seven
six

Writing words.txt

%%file wordfreq.py

import sys

def read_words(file):
    return open(file).read().split()

def wordfreq(words):
    freq = {}
    
    for word in words:
        if word in freq:
            freq[word] += 1
        else:
            freq[word] = 1
    return freq

if __name__ == "__main__":
    words = read_words(sys.argv[1])
    freq = wordfreq(words)
    print(freq)

Writing wordfreq.py

!python wordfreq.py words.txt

{'nine': 1, 'two': 2, 'six': 4, 'four': 4, 'five': 5, 'one': 1, 'seven': 3, 'eight': 2, 'three': 3}

%%file wordfreq.py

import sys

def read_words(file):
    return open(file).read().split()

def wordfreq(words):
    freq = {}
    
    for word in words:
        freq[word] = freq.get(word, 0) + 1
        
    return freq

if __name__ == "__main__":
    words = read_words(sys.argv[1])
    freq = wordfreq(words)
    print(freq)

Overwriting wordfreq.py

!python wordfreq.py words.txt

{'three': 3, 'six': 4, 'nine': 1, 'one': 1, 'eight': 2, 'four': 4, 'two': 2, 'five': 5, 'seven': 3}

import wordfreq

words = wordfreq.read_words("words.txt")

freq = wordfreq.wordfreq(words)

for w, f in freq.items():
    print(w, f)

three 3
nine 1
five 5
two 2
one 1
seven 3
eight 2
four 4
six 4

for k,v in sorted(freq.items()):
    print(k.rjust(5), v)

eight 2
 five 5
 four 4
 nine 1
  one 1
seven 3
  six 4
three 3
  two 2

for k, v in sorted(freq.items(), key = lambda x:x[1]):
    print (k.rjust(5), v)

 nine 1
  one 1
  two 2
eight 2
three 3
seven 3
 four 4
  six 4
 five 5

for k, v in sorted(freq.items(), key = lambda x:x[1], reverse=True):
    print (k.rjust(5), v)

 five 5
 four 4
  six 4
three 3
seven 3
  two 2
eight 2
 nine 1
  one 1

for k, v in sorted(freq.items(), key = lambda x:x[1], reverse=True):
    print (k.rjust(5), v, "*"*v)

 five 5 *****
 four 4 ****
  six 4 ****
three 3 ***
seven 3 ***
  two 2 **
eight 2 **
 nine 1 *
  one 1 *

Grouping all keys based on values

team = {"david":"USA", "anand":"India","linus":"USA","nouful":"India","alice":"UK"}

[name for name in team.keys() if team[name]=="India"]

['anand', 'nouful']

[name for name in team.keys() if team[name]=="USA"]

['david', 'linus']

Pitfalls¶

x = [1,2,3]
y = x
y.append(4)
print(x)

[1, 2, 3, 4]

x = [1, 2, 3, 4]
y = x
y = [1, 2, 3]
print(x)

[1, 2, 3, 4]

x = 1
y = x
y = 2
print(x)

1

Classes¶

class Complex:
    
    def __init__(self, r, i):
        self.real = r
        self.imaginary = i
        
    def get_real(self):
        return self.real
    
    def get_imaginary(self):
        return self.imaginary

p = Complex(10, 5)

class Dummy:
    pass

d = Dummy()

print(d)

<__main__.Dummy object at 0x7fefccee9748>

class Dummy:
    def __init__(self):
        pass

d = Dummy()

print(d)

<__main__.Dummy object at 0x7fefd430d9b0>

p = Complex(10, 5)

type(p)

__main__.Complex

isinstance(p, Complex)

True

isinstance(p, Dummy)

False

class Complex:
    
    def __init__(self, r, i):
        self.real = r
        self.imaginary = i
        
    def get_real(self):
        return self.real
    
    def get_imaginary(self):
        return self.imaginary
    
    def display(self):
        print(self.real, "+", str(self.imaginary) + "j")
        
    def add(self, c):
        r = self.real + c.get_real()
        i = self.imaginary + c.get_imaginary()
        return Complex(r, i)

p = Complex(10, 5)

p1 = Complex(3, 4)

p2 = p.add(p1)

p1.display()
p2.display()
p.display()

3 + 4j
13 + 9j
10 + 5j

Do it yourself¶

add provision to double the complex number

Why classes?¶

%%file bank0.py

balance = 0

def deposit(amount):
    global balance
    balance = balance + amount
    
def withdraw(amount):
    global balance
    balance = balance - amount
    
def get_balance():
    global balance
    return balance

def main():
    deposit(100)
    withdraw(40)
    print(get_balance())
    
    deposit(20)
    print(get_balance())

if __name__ =="__main__":
    main()

Overwriting bank0.py

!python bank0.py

60
80

%%file bank1.py


def make_account():
    return {"balance":0}

def deposit(account , amount):
    account['balance'] = account['balance'] + amount
    
def withdraw(account, amount):
    account['balance'] -= amount
    
def get_balance(account):
    return account['balance']

def main():
    a1 = make_account()
    deposit(a1, 100)
    withdraw(a1, 40)
    print(get_balance(a1))
    
    deposit(a1, 20)
    print(get_balance(a1))
    
    a2 = make_account()
    deposit(a2, 1000)
    withdraw(a2, 200)
    print(get_balance(a2))

if __name__ =="__main__":
    main()

Overwriting bank1.py

!python bank1.py

60
80
800

import time

time.time()

1505900809.371694

Do it yourself¶

Write a class Timer to measure the time taken in task. The class should have start and stop methods and it shoould be able to find time taken between the two calls.hint: use time.time()

t = Timer()
t.start()
do_some_stuff()
t.stop()
print("Time taken: ", t.get_time_taken())

import time
class Timer:
    
    def start(self):
        self.starttime = time.time()
        
    def stop(self):
        self.endtime = time.time()
        
    def get_time_taken(self):
        return self.endtime - self.starttime

t = Timer()
t.start()
s = 0
for i in range(1000):
    for j in range(10000):
        s += i*j
t.stop()
print("Time taken:", t.get_time_taken())

Time taken: 1.8775839805603027

Exceptions¶

z

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-155-3a710d2a84f8> in <module>()
----> 1 z

NameError: name 'z' is not defined

int("Hello")

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-156-5cdea6865089> in <module>()
----> 1 int("Hello")

ValueError: invalid literal for int() with base 10: 'Hello'

"2" * "3"

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-157-c32b222837be> in <module>()
----> 1 "2" * "3"

TypeError: can't multiply sequence by non-int of type 'str'

b = "2"
c = "3"

try:
    a = b*c
except TypeError as e:
    a = 1
    print("Handled TypeError", e)
except ValueError as e:
    b = 0
    print("Handled ValueError", e)

print(a)

Handled TypeError can't multiply sequence by non-int of type 'str'
1

def parseinteger(strnum):
    try:
        return int(strnum)
    except ValueError as e:
        return 0

def read_with_missing(filename):
    with open(filename) as file:
        return [parseinteger(line.strip()) for line in file.readlines()]

%%file missing.txt
1
2
3
4
5
N/A
6
6
7
Nan
8
9

Writing missing.txt

read_with_missing("missing.txt")

[1, 2, 3, 4, 5, 0, 6, 6, 7, 0, 8, 9]

Writing commandline applications¶

%%file command.py 
import argparse

def parse_args():
    p = argparse.ArgumentParser()
    p.add_argument("word", help="word which command.py will print to screen",
                  type=str)
    return p.parse_args()


def print_word(word):
    print(word)

def main():
    args = parse_args()
    print(args)
    print_word(args.word)
    
if __name__ == "__main__":
    main()

Overwriting command.py

!python command.py hello

Namespace(word='hello')
hello

!python command.py

usage: command.py [-h] word
command.py: error: the following arguments are required: word

!python command.py -h

usage: command.py [-h] word

positional arguments:
  word        word which command.py will print to screen

optional arguments:
  -h, --help  show this help message and exit

%%file command.py 
import argparse

def parse_args():
    p = argparse.ArgumentParser()
    p.add_argument("word", help="word which command.py will print to screen",
                  type=str)
    p.add_argument("-u", "--username", 
                  help = "User name for executing this command")
    p.add_argument("--debug", 
               help="Debug mode",
              action="store_true")
    
    return p.parse_args()


def print_word(word):
    print("From default :", word)

def print_user(user):
    print("From -u :", user)
    
def print_debug(debug):
    if debug:
        print("From debug")
    
def main():
    args = parse_args()
    print(args)
    print_word(args.word)
    print_user(args.username)
    print_debug(args.debug)
    
if __name__ == "__main__":
    main()

Overwriting command.py

!python command.py -h

usage: command.py [-h] [-u USERNAME] [--debug] word

positional arguments:
  word                  word which command.py will print to screen

optional arguments:
  -h, --help            show this help message and exit
  -u USERNAME, --username USERNAME
                        User name for executing this command
  --debug               Debug mode

!python command.py -u vikrant --debug hello

Namespace(debug=True, username='vikrant', word='hello')
From default : hello
From -u : vikrant
From debug

!python command.py hello

Namespace(debug=False, username=None, word='hello')
From default : hello
From -u : None

Downloading stuff from internet¶

from urllib.request import urlopen

response = urlopen("http://httpbin.org/html")

response

<http.client.HTTPResponse at 0x7fefd41c7eb8>

contents = response.read()

contents[:100]

b'<!DOCTYPE html>\n<html>\n  <head>\n  </head>\n  <body>\n      <h1>Herman Melville - Moby-Dick</h1>\n\n     '

html = contents.decode("utf-8")

print(html[:400])

<!DOCTYPE html>
<html>
  <head>
  </head>
  <body>
      <h1>Herman Melville - Moby-Dick</h1>

      <div>
        <p>
          Availing himself of the mild, summer-cool weather that now reigned in these latitudes, and in preparation for the peculiarly active pursuits shortly to be anticipated, Perth, the begrimed, blistered old blacksmith, had not removed his portable forge to the hold again, af

response.status

200

There is third party library requests

pip3 install requests

import requests

response = requests.get("http://httpbin.org/html")

print(response.text[:400])

<!DOCTYPE html>
<html>
  <head>
  </head>
  <body>
      <h1>Herman Melville - Moby-Dick</h1>

      <div>
        <p>
          Availing himself of the mild, summer-cool weather that now reigned in these latitudes, and in preparation for the peculiarly active pursuits shortly to be anticipated, Perth, the begrimed, blistered old blacksmith, had not removed his portable forge to the hold again, af

response.headers

{'Via': '1.1 vegur', 'Content-Type': 'text/html; charset=utf-8', 'Connection': 'keep-alive', 'X-Powered-By': 'Flask', 'Server': 'meinheld/0.6.1', 'Date': 'Wed, 20 Sep 2017 11:22:44 GMT', 'X-Processed-Time': '0.000496864318848', 'Access-Control-Allow-Credentials': 'true', 'Access-Control-Allow-Origin': '*', 'Content-Length': '3741'}

response.status_code

200

response = requests.get("http://httpbin.org/get", params={"param1":"hello", "param2":"hello2"})

print(response.text)

{
  "args": {
    "param1": "hello", 
    "param2": "hello2"
  }, 
  "headers": {
    "Accept": "*/*", 
    "Accept-Encoding": "gzip, deflate", 
    "Connection": "close", 
    "Host": "httpbin.org", 
    "User-Agent": "python-requests/2.18.4"
  }, 
  "origin": "42.106.10.34", 
  "url": "http://httpbin.org/get?param1=hello&param2=hello2"
}

response = requests.post("http://httpbin.org/post", data="some string")

print(response.text)

{
  "args": {}, 
  "data": "some string", 
  "files": {}, 
  "form": {}, 
  "headers": {
    "Accept": "*/*", 
    "Accept-Encoding": "gzip, deflate", 
    "Connection": "close", 
    "Content-Length": "11", 
    "Host": "httpbin.org", 
    "User-Agent": "python-requests/2.18.4"
  }, 
  "json": null, 
  "origin": "42.106.10.34", 
  "url": "http://httpbin.org/post"
}

Find popular repositories of vmware on github¶

url = "https://api.github.com/orgs/vmware/repos"

repos = requests.get(url).json()

type(repos)

list

for repo in repos:
    print(repo['full_name'], repo['forks'])

vmware/pyvco 4
vmware/rvc 46
vmware/rbvmomi 152
vmware/vprobe-toolkit 8
vmware/CloudFS 15
vmware/vcd-nclient 2
vmware/lmock 5
vmware/FireBreath 2
vmware/weasel 1
vmware/vmware-vcenter 83
vmware/vmware-vshield 6
vmware/vcloud-rest 38
vmware/GemstoneWebTools 0
vmware/vmware-vcsa 17
vmware/vmware-vmware_lib 23
vmware/saml20serviceprovider 1
vmware/pg_rewind 19
vmware/vco-powershel-plugin 2
vmware/jenkins-reviewbot 12
vmware/dbeekeeper 0
vmware/thinapp_factory 16
vmware/vmware-cassandra 4
vmware/vmware-java 0
vmware/data-driven-framework 2
vmware/pyvmomi 419
vmware/pyvmomi-community-samples 352
vmware/open-vm-tools 132
vmware/pyvmomi-tools 18
vmware/upgrade-framework 11
vmware/webcommander 29

reps = sorted(repos, key=lambda r:r['forks'], reverse=True)[:5]

for r in reps:
    print(r['full_name'], r['forks'])

vmware/pyvmomi 419
vmware/pyvmomi-community-samples 352
vmware/rbvmomi 152
vmware/open-vm-tools 132
vmware/vmware-vcenter 83

Feedback please¶

feedback