Python Virtual Training For Arcesium - Module III - Day 2¶

Aug 19-25, 2022 Vikrant Patil

All notes are available online at https://notes.pipal.in/2022/arcesium_finop_batch1/

Please accept the invitation that you have received in your email and login to

https://engage.pipal.in/

login to lab and create today's notebook module3-day2

© Pipal Academy LLP

Virtual Environment¶

Many times it happens that different projects have requirements of python packages such that they conflict each other. In such cases how do you work on two different project on same machine? If we install python packages for one project, those packages will confict with other projects. Virtual environment is there to help us. Virtual environment allows us to have set of python packages seperately for each project. Also added advantage is, it won't affect system python's packages. The way to handle this is with help of venv module we create virtual environment for each project. All requirements for the proejct are installed in the virtual environment and not in system python's packages. Let's take some examples.

Conflicting Requirements¶

Suppose we have two projects, datascraping and analytics. For datascraping project requirements are following packages

  requests==2.24.0
  openpyxl==2.4.8

and analytics project needs following packages

  pandas==1.1.2
  openpyxl==3.0.5
  requests==2.24.0

Now here is confiliting requirement, one project needs openpyxl verson 2.4.8 and other needs 3.0.5.

creating evenv¶

To create virutal environment on your system, what you need is python version > 3.5. Python comes with a package called venv (virtual environment). For older python, virtualenv was seperate application. We are going to work with virtual environment that comes with python 3. Easy steps to work with it are as given below. Open up terminal on linux/mac or cmd terminal on windows. on the prompt type following command to create virtual environment with name env1


  python -m venv env1

This will create a folder with name env1 in the current directory. On linux it will have following contents

  +-env1
    |
    +-bin
    +-include
    +-lib
    +-lib64
    +-pyenv.cfg

on windows system it will have following contents

  +-env1
    |
    +-Include
    +-Lib
    +-Scripts
    +-pyenv.cfg

To activate virtual environment on linux run following command on terminal.

  bash$ source env1/bin/activate
  (env1) bash$ # you can see the env1 environment activated as change in prompt

To activate virtual environment on windows run following command on windows cmd terminal


  C:\Users\vik> env1\bin\activate.bat
  (env1) C:\Users\vik>

Installing packages in virtual environment¶

Once the virtul environment is created and activated, we are ready to use it. To install packages in this active virtual environment use pip install

  pip install typer
  Collecting typer
    Using cached https://files.pythonhosted.org/packages/90/34/d138832f6945432c638f32137e6c79a3b682f06a63c488dcfaca6b166c64/typer-0.3.2-py3-none-any.whl
  Collecting click<7.2.0,>=7.1.1 (from typer)
    Using cached https://files.pythonhosted.org/packages/d2/3d/fa76db83bf75c4f8d338c2fd15c8d33fdd7ad23a9b5e57eb6c5de26b430e/click-7.1.2-py2.py3-none-any.whl
  Installing collected packages: click, typer
  Successfully installed click-7.1.2 typer-0.3.2

to check packages installed


  pip list
  Package    Version
  ---------- -------
  click      7.1.2
  pip        19.2.3
  setuptools 41.2.0
  typer      0.3.2

requirements.txt¶

If we want to replicate exact same virtual environment on other machine we need list of packages that pip can understand. The format is called as requirements file. it can be generated using

  pip freeze
  click==7.1.2
  typer==0.3.2

The output can be saved to a file with name requirements.txt. This file can be used in other virtual env to recreate the same environment. For example, lets make use of above requirements to recreate another environment with name env1copy

  bash$ python -m venv env1copy
  bash$ source env1copy/bin/activate
  (env1copy) bash$ pip install -r requirements.txt
  Collecting click==7.1.2 (from -r env1/requirements.txt (line 1))
    Using cached https://files.pythonhosted.org/packages/d2/3d/fa76db83bf75c4f8d338c2fd15c8d33fdd7ad23a9b5e57eb6c5de26b430e/click-7.1.2-py2.py3-none-any.whl
  Collecting typer==0.3.2 (from -r env1/requirements.txt (line 2))
    Using cached https://files.pythonhosted.org/packages/90/34/d138832f6945432c638f32137e6c79a3b682f06a63c488dcfaca6b166c64/typer-0.3.2-py3-none-any.whl
  Installing collected packages: click, typer
  Successfully installed click-7.1.2 typer-0.3.2

you can check the packages installed

  pip freeze
  click==7.1.2
  typer==0.3.2

Summary¶

  1. Virtual environment can be created by any user. No admin privileges required.
  2. Every virtual environment is stored in seperate folder.
  3. Packages installed in a virtual environment are only in that particular virtual environment.
  4. With requirements.txt it is very easy to recreate the same replica of a particular virtul environment.
In [ ]:

Working json data¶

In [1]:
pydata = {"values":[1, 2, 3, 3, 4],
          "symbols":["X","Y","Z","A","B"]}
In [2]:
pydata
Out[2]:
{'values': [1, 2, 3, 3, 4], 'symbols': ['X', 'Y', 'Z', 'A', 'B']}
In [3]:
import json
In [9]:
json.dumps(pydata) # converting python data to json format
Out[9]:
'{"values": [1, 2, 3, 3, 4], "symbols": ["X", "Y", "Z", "A", "B"]}'
In [5]:
jsondata = json.dumps(pydata)
In [6]:
type(jsondata)
Out[6]:
str
In [8]:
json.loads(jsondata) # converting back json data to python data
Out[8]:
{'values': [1, 2, 3, 3, 4], 'symbols': ['X', 'Y', 'Z', 'A', 'B']}

Downloading data from internet¶

In [11]:
url = "https://raw.githubusercontent.com/vikipedia/python-trainings/master/online_course/source/module2/wallet.csv"
In [12]:
import requests

/home/vikrant/usr/local/jupyter-py3.10/lib/python3.10/site-packages/requests/__init__.py:102: RequestsDependencyWarning: urllib3 (1.26.9) or chardet (5.0.0)/charset_normalizer (2.0.12) doesn't match a supported version!
  warnings.warn("urllib3 ({}) or chardet ({})/charset_normalizer ({}) doesn't match a supported "
In [13]:
!pip install requests

Requirement already satisfied: requests in /home/vikrant/usr/local/jupyter-py3.10/lib/python3.10/site-packages (2.27.1)
Requirement already satisfied: idna<4,>=2.5 in /home/vikrant/usr/local/jupyter-py3.10/lib/python3.10/site-packages (from requests) (3.3)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/vikrant/usr/local/jupyter-py3.10/lib/python3.10/site-packages (from requests) (1.26.9)
Requirement already satisfied: charset-normalizer~=2.0.0 in /home/vikrant/usr/local/jupyter-py3.10/lib/python3.10/site-packages (from requests) (2.0.12)
Requirement already satisfied: certifi>=2017.4.17 in /home/vikrant/usr/local/jupyter-py3.10/lib/python3.10/site-packages (from requests) (2021.10.8)

[notice] A new release of pip available: 22.1.2 -> 22.2.2
[notice] To update, run: pip install --upgrade pip
In [14]:
response = requests.get(url)
In [16]:
response.status_code # if the status is 200 , then request was suscessfull
Out[16]:
200
In [17]:
type(response.content)
Out[17]:
bytes
In [18]:
type(response.text)
Out[18]:
str
In [19]:
response.text[:200]
Out[19]:
',date,category,description,debit\n0,2021-03-07 14:53:28.377359,Music,Amazon,421.2073272347991\n1,2020-10-08 09:53:28.377359,Food,Swiggy,328.4400802428426\n2,2021-02-23 09:53:28.377359,Books,Amazon,244.67'
In [20]:
with open("datafrom_internet.csv", "w") as f: # open a file in write mode, by default opens in a text mode!
    f.write(response.text)
In [21]:
!head datafrom_internet.csv

,date,category,description,debit
0,2021-03-07 14:53:28.377359,Music,Amazon,421.2073272347991
1,2020-10-08 09:53:28.377359,Food,Swiggy,328.4400802428426
2,2021-02-23 09:53:28.377359,Books,Amazon,244.67943701511354
3,2020-11-01 14:53:28.377359,Utility,Phone,222.7563175805277
4,2021-06-05 13:53:28.377359,Books,Flipcart,494.1284923793595
5,2021-07-28 19:53:28.377359,Utility,Electricity,219.94171130968408
6,2021-04-16 11:53:28.377359,Books,Amazon Kindle,270.32259514795845
7,2021-02-15 10:53:28.377359,Food,Zomato,457.1831036346536
8,2021-08-10 19:53:28.377359,Utility,Phone,151.49637259947792
In [22]:
excelurl = "https://raw.githubusercontent.com/vikipedia/python-trainings/master/online_course/source/module2/wallet.xlsx"
In [23]:
excelresponse = requests.get(excelurl)
In [24]:
excelresponse.status_code
Out[24]:
200
In [25]:
excelresponse.text[:10]
Out[25]:
'PK\x03\x04\x14\x00\x08\x08\x08\x00'
In [31]:
with open("exceldata1.xlsx", "wb") as fb: # opened file in brinary mode and write mode
    fb.write(excelresponse.content)
In [32]:
import pandas
In [33]:
type(excelresponse.content) # binary
Out[33]:
bytes
In [34]:
type(excelresponse.text)
Out[34]:
str
In [36]:
with open("exceltext.xlsx", "w") as f:
    f.write(excelresponse.text) # this will corrupt the data!
In [37]:
help(bin)

Help on built-in function bin in module builtins:

bin(number, /)
    Return the binary representation of an integer.
    
    >>> bin(2796202)
    '0b1010101010101010101010'
In [38]:
bin(234)   "234"
Out[38]:
'0b11101010'
In [39]:
bin(22)
Out[39]:
'0b10110'
In [40]:
bin("22")

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Input In [40], in <cell line: 1>()
----> 1 bin("22")

TypeError: 'str' object cannot be interpreted as an integer
In [41]:
help(ord)

Help on built-in function ord in module builtins:

ord(c, /)
    Return the Unicode code point for a one-character string.
In [43]:
ord("2")
Out[43]:
50
In [44]:
bin(50)
Out[44]:
'0b110010'
In [ ]: