Simeon Simeonov
Present the Python programming language in a different way than https://docs.python.org
Avoid information overload
Use examples and interaction rather than documents and slides
Basics: About the language, the Python eco-system, types, modules, functions, scopes, decorators, string formatting
Object-oriented programming in Python: How Python "really works"
Control flow: if / for / while / try, iterators, "tactical programming" tips
A brief tour through Python's standard library
Code design and best practices: How to design your code
Python is an interpreted high-level general-purpose programming language - advanced through the Python Enhancement Proposal (PEP) process
CPython is the reference implementation of Python, written in C (alternatives: pypy, jython)
python - interpreter and interpreter shell (alternatives: ipython, bpython)
libpython
Calling C from Python: Cython, CFFI, ctypes
$ python
import this
# The Zen of Python, by Tim Peters
# Beautiful is better than ugly.
# Explicit is better than implicit.
# Simple is better than complex.
# Complex is better than complicated.
# Flat is better than nested.
# Sparse is better than dense.
# Readability counts.
# Special cases aren't special enough to break the rules.
# Although practicality beats purity.
# Errors should never pass silently.
# Unless explicitly silenced.
# In the face of ambiguity, refuse the temptation to guess.
# There should be one-- and preferably only one --obvious way to do it.
# Although that way may not be obvious at first unless you're Dutch.
# Now is better than never.
# Although never is often better than *right* now.
# If the implementation is hard to explain, it's a bad idea.
# If the implementation is easy to explain, it may be a good idea.
# Namespaces are one honking great idea -- let's do more of those!
Few built-in functions.
https://docs.python.org/3/library/functions.html
Python uses duck typing and has typed objects but untyped variable names.
Type constraints are not checked at compile time; rather, operations on an object may fail, signifying that the given object is not of a suitable type. Despite being dynamically-typed, Python is strongly-typed, forbidding operations that are not well-defined (for example, adding a number to a string) rather than silently attempting to make sense of them.
s = 'foo' # this is a string / str, same as str('foo'), may be encoded, immutable
b = b'foo' # bytes, same as bytes('foo', 'utf-8'), may be decoded, immutable
i = 6 # int, same as int('6'), immutable
f = 0.1 # float, same as float('0.1'), immutable
b = False # bool, same as bool(0), bool(''), bool(None)... immutable / constant
n = None # NoneType, similar to 'null' in other languages, immutable / constant
l = [1, False, 'foo'] # list, same as list((1, False, 'foo'))
t = (1, False, 'foo') # tuple, same as tuple([1, False, 'foo']), immutable
d = {'foo': 1, 'bar': 8} # dict, same as dict(foo=1, bar=8), similar to hash in other languages
s = {'foo', 'bar', 1, 1, 4} # set, same as set(['foo', 'bar', 1, 1, 4]), removes duplicates
Classes, functions, modules, .... and even types are simply other types :)
A module is a file containing Python definitions and statements. The file name is the module name with the suffix .py appended. Within a module, the module's name (as a string) is available as the value of the global variable __name__
When a module named foo is imported, the interpreter first searches for a built-in module with that name (sys.builtin_module_names). If not found, it then searches for a file named foo.py in a list of directories given by the variable sys.path. sys.path is initialized from these locations:
The module is then imported only once and "cached" in sys.modules
Packages are a way of structuring Python's module namespace by using "dotted module names"
The import statement combines two operations:
# bar.py then bar/__init__.py will be considered, the first match executed and bound to 'bar'
import bar
import mymodule.foo # implicitly executes mymodule.py, mymodule/__init__.py and mymodule/foo/__init__.py
import numpy as np # will be bound as 'np' instead of 'numpy'. N.B. __name__ is still 'numpy'
import some.extremely.deep.path.Animal as Animal # "sacrifice" the namespace in the name of convinience
from sys import path # execute sys and only import the 'path' attribute into local scope as 'path'
# relative imports must be explicit in Python 3
from .othermodule import something # expects that current module and 'othermodule' are in the same
# package (containing __init__.py)
from sys import * # NO! Bad programming practice since 1879
Python's official package repository is PyPi (https://pypi.org), while Python's official package installer is pip (https://pypi.org/project/pip/)
A Python environment is the physical and logical arrangement of Python modules and packages. Several options exist:
Desired qualities for a flexible Python environment:
Exploting the operating system can be done by:
Using venv can be done by directly invoking python:
# create a virtual environment
python -m venv my_virtual_env
python -m venv --system-site-packages my_virtual_env
# load, use and unload the virtual environment
source my_virtual_env/bin/activate
pip install sqlalchemy
# install package from a custom repository (https://artifactory.fifty.eu)
pip install --index-url=https://artifactory.fifty.eu/artifactory/api/pypi/pypi/simple/ odin-data-access
deactivate
# one can alternatively use the python "wrapper" of the virtual env
my_virtual_env/bin/python -m pip install sqlalchemy
--system-site-packages will keep the original site-packages folders at the end of sys.path
Poetry (https://python-poetry.org) is the prefered environment and dependency management tool at Statnett.
# create project and a virtual environment from scratch
poetry new my-project
# ... or use Poetry with an existing one
cd my-project
poetry init
# edit pyproject.toml for your needs (f.i. add dependencies, metadata ... etc),
# create virtual environment and install dependencies
poetry install
# finally commit your poetry.lock file to version control
# update all dependencies
poetry update
For more info: https://python-poetry.org/docs/basic-usage/
Immutable object is an object with a fixed value. Immutable objects include bool, int, float, str, bytes and tuples. Such an object cannot be altered. A new object has to be created if a different value has to be stored. They play an important role in places where a constant hash value is needed, for example as a key in a dictionary.
All objects that are not immutable are... mutable. All hashable objects should be immutable or use id().
i = 1
id(i) # returns f.i. 9788992
i += 1 # same as i = i + 1
id(i) # returns a different value, hence - a brand new object
s = 'Hello'
s += ' World' # s is now a different object
s[0] # 'H'
s[0] = 'h' # TypeError: 'str' object does not support item assignment
t = (1, 4) # tuple
l = [1, 4] # list
hash(t) # returns f.i. -6333845781340707986
hash(l) # TypeError: unhashable type: 'list'
s = 'the long and winding road'
s2 = 'the long and winding road'
# check if s and s2 are the same object:
id(s) # Out: 139858905258704
id(s2) # Out: 139858926037472
# the hash should be the same
hash(s) # Out: 7030216208569256362
hash(s2) # Out: 7030216208569256362
A function is a sequence of program instructions that performs a specific task, packaged as a unit.
Functions let you:
Function calls bring some overhead pushing / popping function-data into / from stack.
Important definitions (may have different meanings in different programming languages):
The keyword def introduces a function definition.
def add(a, b): # - function definition / header
"""Function for adding integers""" # - docstring
result = a + b
a = 5
return result # - function that does not contain return, implicitly returns None
a_param = 9
b_param = -2
add(a_param, b_param) # Out: 7
# integers are immutable and a_param will remain unchanged
print(a_param) # Out: 9
def addl(a, b):
"""Function for adding two lists"""
result = a + b
a += [5] # in this case equal to: a.append(5)
return result
a_param = [9]
b_param = [-2]
addl(a_param, b_param) # Out: [9, -2]
# lists are mutable and a_param will be changed
print(a_param) # Out: [9, 5]
def add(a, b):
"""Function for adding integers"""
return a + b
my_result = add(2, 5) # positional arguments (parameters)
my_result = add(b=5, a=2) # keyword arguments (parameters)
my_tuple = (2, 5)
my_dict = {'b': 5, 'a': 2}
my_result = add(*my_tuple) # unpacked and assigned to the positional arguments
my_result = add(**my_dict) # unpacked and assigned to the kw. arguments
def add(a, b=5):
"""Function for adding integers"""
return a + b
my_result = add(2)
# ... and the rest of the examples above will work
def add(a, *args, **kwargs):
"""Function for adding integers"""
if args:
b = args[0]
elif 'b' in kwargs:
b = kwargs['b']
return a + b
my_result = add(2, 5, 9, 11) # 5 assigned to args[0]
my_result = add(2, b=5, c=9, d=11) # 5 assigned to kwargs['b']
my_result = add(2, b=5, 9, 11) # SyntaxError: positional argument follows keyword argument
def decrypt(password: str, edata: str) -> str:
"""
Decrypts `edata` using `password`.
`edata` is in the following format:
enc-val$`version-num`$`bas64-salt`$`base64-encrypted_data`
:param password: The password to generate the key with
:type password: str
:param edata: The data to be decrypted
:type edata: str
:raises EtoolkitInstanceError: If the encryption format is unsupported
:return: The output string / decrypted data
:rtype: str
"""
if not edata.startswith('enc-val$1$'):
raise EtoolkitInstanceError('Unsupported encryption format')
# some more code magic coming after....
# ...
# ..
return decrypted_str
import typing
Basestring = typing.Union[str, bytes]
def decrypt(password: Basestring, edata: str) -> str:
pass
from typing import Union
def decrypt(password: Union[str, bytes], edata: str) -> str:
"""Generic documentation. No need for pass"""
# Python >= 3.10 only
def decrypt(password: str | bytes, edata: str) -> str:
"""Generic documentation. No need for pass"""
def fetch_the_first_letter(input_str: str) -> str:
"""Fetches the first letter of the string input_str or 'x'""""
try:
return input_str[0]
except Exception:
return 'x'
letter_list = list(map(fetch_the_first_letter, ['foo', 'bar', 'test'])) # Out: ['f', 'b', 't']
Small anonymous functions can be created with the lambda keyword
letter_list = list(map(lambda x: x[0], ['foo', 'bar', 'test'])) # Out: ['f', 'b', 't']
Functions in Python are callable objects. Callable objects can be created by defining the __call__ method. More on that later in the course...
A function can return multiple values by implicitly returning a tuple:
def square_cube(x):
"""returns x, x^2 and x^3"""
return x, x**2, x**3
numbers = square_cube(5) # Out: (5, 25, 125)
num, sqnum, cbnum = square_cube(5) # unpacking the tuple
Can be used as:
def get_multiplier_of(base: int) -> str:
"""the function enclosing its nested functions"""
def multiplier_function(x):
"""a nested function"""
return base * x
return multiplier_function
times3 = get_multiplier_of(3)
times5 = get_multiplier_of(5)
print(times3(3)) # Out: 9
print(times5(3)) # Out: 15
locals() and globals() return dicts of symbols for their respective scopes
num1, num2 = 7, 8 # module globals
def print_numbers():
print(num1, num2) # OK, these are module globals
num3 = 100
print(num3) # prints 100, num3 is in the function (local) scope
global num4 # assignes / references num4 to / in the global scope
num4 = 99
id = 200 # new symbol in local scope
# id(num4) # will not yield the expected result (raises TypeError)
def print_numbers2():
print(num3) # OK, enclosed scope
print_numbers2() # prints 100
print_numbers()
# print(num3) # Raises NameError - why?
print(num4) # Prints 99 - why?
# print_numbers2() # Raises NameError
Decorators can be used to modify the behavior of the objects they decorate. Decorators can be implemented either by using classes or by using nested functions.
def my_decorator(func):
def decorated():
print('Doing something before the decorated function')
retval = func()
print('Doing something after the decorated function')
return retval
return decorated
def my_function():
print('Alice')
my_function = my_decorator(my_function)
my_function()
... may be dificult to read / understand, while:
@my_decorator
def my_function():
print('Alice')
my_function()
... may be easier
import sys
from functools import wraps
def requires_access(access_secret: str):
def api_access_decorator(f):
@wraps(f)
def decorated(*args, **kwargs):
if 'secret' not in kwargs:
sys.exit('No secret provided')
if not kwargs['secret'] or kwargs['secret'] != access_secret:
sys.exit("Secret doesn't match")
# return f(args[0], **kwargs)
return f(*args, **kwargs)
return decorated
return api_access_decorator
@requires_access(access_secret='b28cfeaa65b73cf')
def sensitive_function(data, **kwargs):
"""very sensitive function"""
db.save(data)
f = 6.57865
i = 27
s = 'another string'
'%s - %d - %5.2f' % (s, i, f) # Out: 'another string - 27 - 6.58'
# still used in:
logger.debug("%d - %s", event.id, message)
'{} - {} - {:5.2f}'.format(s, i, f) # implicit
'{0} - {1} - {2:5.2f}'.format(s, i, f) # explicit
'{my_str} - {i} - {fl:5.2f}'.format(my_str=s, fl=f, i=i) # keyword
# Out: 'another string - 27 - 6.58'
# modern Python >= 3.6 f-strings
f'{s} - {i} - {f:5.2f}' # Out: 'another string - 27 - 6.58'
See https://docs.python.org/3/library/string.html#formatspec for the complete format specification