{ "cells": [ { "cell_type": "markdown", "id": "637cdd40-65ae-43e5-8add-d60b89a5846c", "metadata": {}, "source": [ "# Introduction to Python\n", "\n", "\n", "## Goals\n", "\n", "- Present the Python programming language in a different way than [https://docs.python.org](https://docs.python.org)\n", "\n", "- Avoid information overload\n", "\n", "- Use examples and interaction rather than documents and slides\n", "\n", "\n", "## Preliminary plan\n", "\n", "- **Basics: About the language, the Python eco-system, types, modules, functions, scopes, decorators, string formatting**\n", "\n", "- Object-oriented programming in Python: How Python \"really works\"\n", "\n", "- Control flow: if / for / while / try, iterators, \"tactical programming\" tips\n", "\n", "- A brief tour through Python's standard library\n", "\n", "- Code design and best practices: How to design your code" ] }, { "cell_type": "markdown", "id": "5a72ab1d-f313-4f5f-93c7-60c1e07f8c22", "metadata": {}, "source": [ "## What is Python?\n", "\n", "- Python is an interpreted high-level general-purpose programming language - advanced through the Python Enhancement Proposal (PEP) process\n", "\n", "- CPython is the reference implementation of Python, written in C (alternatives: pypy, jython)\n", "\n", "- python - interpreter and interpreter shell (alternatives: ipython, bpython)\n", "\n", "- libpython\n", "\n", "- Calling C from Python: Cython, CFFI, ctypes" ] }, { "cell_type": "markdown", "id": "6d1edcfc-b001-4393-9c24-72adc9b5fccf", "metadata": {}, "source": [ "## Philosophy\n", "\n", "```python\n", "import this\n", "```" ] }, { "cell_type": "code", "execution_count": 50, "id": "54c9e132-30b0-41bf-97e9-0e8b94f3d202", "metadata": {}, "outputs": [], "source": [ "import this" ] }, { "cell_type": "markdown", "id": "4f144f73-51c1-419d-b71e-ee513f68f41c", "metadata": {}, "source": [ "## Built-in functions\n", "\n", "Few built-in functions.\n", "\n", "[https://docs.python.org/3/library/functions.html](https://docs.python.org/3/library/functions.html)\n", "\n", "- `dir([obj])` - returns a list of valid attributes for that object\n", "\n", "- `id(obj)` - returns the \"identity\" of an object - an integer which is guaranteed to be unique\n", "\n", "- `print(...)` - prints objects to a text stream\n", "\n", "- `str(...)` - returns a string version of object\n", "\n", "- `type(obj)` - returns the type of an object" ] }, { "cell_type": "markdown", "id": "a0c57535-e285-47af-80f9-1d4a16de5f25", "metadata": {}, "source": [ "## Common built-in types\n", "\n", "Python uses duck typing and has typed objects but untyped variable names.\n", "\n", "Type constraints are not checked at compile time; rather, operations on an object may fail, signifying that the given object is not of a suitable type.\n", "Despite being dynamically-typed, Python is strongly-typed, forbidding operations that are not well-defined (for example, adding a number to a string) rather than silently attempting to make sense of them.\n", "\n", "Variables / values must be of a certain type (class)." ] }, { "cell_type": "code", "execution_count": 51, "id": "bd6134d9-cc0e-4b45-b6b3-50210f6395e2", "metadata": {}, "outputs": [], "source": [ "# Common built-in types:\n", "\n", "s = 'foo' # this is a string / str, same as str('foo'), may be encoded, immutable (s[0] = 'r' is NOT possible)\n", "\n", "b = b'foo' # bytes, same as bytes('foo', 'utf-8'), may be decoded, immutable\n", "\n", "i = 6 # int, same as int('6'), immutable\n", "\n", "f = 0.1 # float, same as float('0.1'), immutable, Note!!: Floats have a fixed size,\n", "# hence they don't necessarily behave they way we expect from math class.\n", "\n", "b = False # bool, same as bool(0), bool(''), bool(None)... immutable / constant\n", "\n", "n = None # NoneType, similar to 'null' in other languages, immutable / constant\n", "\n", "l = [1, False, 'foo'] # list, same as list((1, False, 'foo'))\n", "\n", "t = (1, False, 'foo') # tuple, same as tuple([1, False, 'foo']), immutable\n", "\n", "d = {'foo': 1, 'bar': 8} # dict, same as dict(foo=1, bar=8), similar to hash in other languages\n", "\n", "s = {'foo', 'bar', 1, 1, 4} # set, same as set(['foo', 'bar', 1, 1, 4]), removes duplicates" ] }, { "cell_type": "code", "execution_count": 52, "id": "cd2b7f98-ea16-49e2-9d19-f37266bd4f3a", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "'can be written like this'\n" ] }, { "data": { "text/plain": [ "'foo bar'" ] }, "execution_count": 52, "metadata": {}, "output_type": "execute_result" }, { "name": "stdout", "output_type": "stream", "text": [ "works just fine\n", "works\n", "just\n", "fine\n" ] }, { "data": { "text/plain": [ "'S'" ] }, "execution_count": 52, "metadata": {}, "output_type": "execute_result" }, { "data": { "text/plain": [ "'t'" ] }, "execution_count": 52, "metadata": {}, "output_type": "execute_result" }, { "data": { "text/plain": [ "'tatnett'" ] }, "execution_count": 52, "metadata": {}, "output_type": "execute_result" }, { "data": { "text/plain": [ "'tatnet'" ] }, "execution_count": 52, "metadata": {}, "output_type": "execute_result" }, { "data": { "text/plain": [ "'ett'" ] }, "execution_count": 52, "metadata": {}, "output_type": "execute_result" }, { "data": { "text/plain": [ "'tte'" ] }, "execution_count": 52, "metadata": {}, "output_type": "execute_result" }, { "data": { "text/plain": [ "True" ] }, "execution_count": 52, "metadata": {}, "output_type": "execute_result" }, { "data": { "text/plain": [ "False" ] }, "execution_count": 52, "metadata": {}, "output_type": "execute_result" }, { "data": { "text/plain": [ "True" ] }, "execution_count": 52, "metadata": {}, "output_type": "execute_result" }, { "data": { "text/plain": [ "b'B\\xc3\\x98!'" ] }, "execution_count": 52, "metadata": {}, "output_type": "execute_result" }, { "data": { "text/plain": [ "'BØ!'" ] }, "execution_count": 52, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Strings and bytes\n", "\n", "# apart from other programming languages (PHP, Perl), the next two lines are virtually equal in Python\n", "s = 'can be written like this' # ... and should be written like this\n", "s = \"can be written like this\"\n", "print(repr(s)) # the official Python repr uses single quotes\n", "\n", "# the only important difference:\n", "s = 'can\\'t be written like this'\n", "s = \"can't be written like this\" # better\n", "s = \"can be written \\\"like this\\\"\"\n", "s = 'can be written \"like this\"' # better\n", "\n", "s1 = 'foo'\n", "s2 = 'bar'\n", "s1 + ' ' + s2\n", "\n", "# without new line\n", "print(\n", " 'works '\n", " 'just '\n", " 'fine'\n", ")\n", "\n", "# with new line\n", "print(\n", " 'works\\n'\n", " 'just\\n'\n", " 'fine'\n", ")\n", "\n", "# slicing [from:until(not included):step] - produces a new string (copy)\n", "s3 = 'Statnett'\n", "s3[0] # Out: S\n", "s3[-1] # Out: t\n", "s3[1:] # Out: tatnett\n", "s3[1:-1] # Out: tatnett\n", "s3[-3:] # Out: ett\n", "s3[1:-1:2] # Out: tte\n", "\n", "s3.endswith('nett') # Out: True\n", "s3.startswith('sta') # Out: False (case sensitive)\n", "s3[-4:] == 'nett' # works, but BAD coding style\n", "\n", "# All strings in Python 3 are unicode strings (unicode type) and not byte strings as in Python 2\n", "# One doesn't write strings to: files, sockets, cryptographic functions etc... but rather 'bytes'\n", "\n", "# from unicode string to bytes (UTF-8)\n", "'BØ!'.encode('utf-8') # utf-8 is implicit in Python 3\n", "\n", "# bytes to unicode\n", "b'B\\xc3\\x98!'.decode('utf-8') # utf-8 is implicit in Python 3" ] }, { "cell_type": "markdown", "id": "6b2bc8fa-a5dc-4a58-92b2-abfb19bfe2ed", "metadata": {}, "source": [ "## Modules\n", "\n", "A module is a file containing Python definitions and statements. The file name is the module name with the suffix *.py* appended. Within a module, the module's name (as a string) is available as the value of the global variable `__name__`\n", "\n", "When a module named *\"foo\"* is imported, the interpreter first searches for a built-in module with that name (`sys.builtin_module_names`). If not found, it then searches for a file named *foo.py* in a list of directories given by the variable `sys.path`. `sys.path` is initialized from these locations:\n", "\n", "- the directory containing the input script (or the current directory when no file is specified)\n", "\n", "- *PYTHONPATH* - env. variable - a list of directory names\n", "\n", "- the installation-dependent default locations\n", "\n", "The module is then imported only once and \"cached\" in `sys.modules`\n" ] }, { "cell_type": "markdown", "id": "6c55e90b-9385-4ebd-90e9-e29654096287", "metadata": {}, "source": [ "## Packages\n", "\n", "Packages are a way of structuring Python's module namespace by using \"dotted module names\"\n", "\n", "The import statement combines two operations:\n", "\n", "- it searches for the named module\n", "\n", "- it binds the results of that search to a name in the *local scope*\n", "\n", "\n", "```python\n", "# bar/__init__.py then bar.py will be considered, the first match executed and bound to 'bar'\n", "import bar\n", "\n", "import mymodule.foo # implicitly executes mymodule/__init__.py (or mymodule.py) and mymodule/foo/__init__.py\n", "\n", "import numpy as np # will be bound as 'np' instead of 'numpy'. N.B. __name__ is still 'numpy'\n", "\n", "import some.extremely.deep.path.Animal as Animal # \"sacrifice\" the namespace in the name of convinience\n", "\n", "from sys import path # execute sys and only import the 'path' attribute into local scope as 'path'\n", "\n", "# relative imports must be explicit in Python 3\n", "from .othermodule import something # expects that current module and 'othermodule' are in the same\n", " # package (containing __init__.py)\n", "\n", "from sys import * # NO! Bad programming practice since 1879\n", "```" ] }, { "cell_type": "markdown", "id": "b26c4bd1-58ea-44bc-aedd-e6b6405da015", "metadata": {}, "source": [ "## Creating and maintaining a Python environment\n", "\n", "Python's official package repository is PyPi [https://pypi.org](https://pypi.org), while Python's official package installer is *pip* [https://pypi.org/project/pip/](https://pypi.org/project/pip/)\n", "\n", "A Python environment is the physical and logical arrangement of Python modules and packages. Several options exist:\n", "\n", "- using a proper operating system :) (symlinks, real commercial support etc.)\n", "\n", "- using venv\n", "\n", "- using higher level tools like *poetry*\n", "\n", "- using a mixture / cocktail of all of the above :)" ] }, { "cell_type": "markdown", "id": "40db50e4-fb7a-4a77-a074-bc4413f27212", "metadata": {}, "source": [ "## Creating and maintaining a Python environment (cont...)\n", "\n", "Desired qualities for a flexible Python environment:\n", "\n", "- easy to create and (un)load\n", "\n", "- do not require extra privileges\n", "\n", "- don't repeat yourself (DRY)\n", "\n", "- easy to update without breaking the API\n", "\n", "- easy to debug\n", "\n", "- play nicely with the VCS (git)" ] }, { "cell_type": "markdown", "id": "c661bf56-add0-402c-a629-a857f81d0758", "metadata": {}, "source": [ "## Creating and maintaining a Python environment (cont...)\n", "\n", "Exploting the operating system can be done by:\n", "\n", "- (re)defining `PYTHONPATH`\n", "- using symlinks to point at packages placed at different locations" ] }, { "cell_type": "markdown", "id": "db8952aa-0944-4644-9626-7460786fd72a", "metadata": {}, "source": [ "## Creating and maintaining a Python environment (cont...)\n", "\n", "Using venv can be done by directly invoking python:\n", "\n", "```bash\n", "# create a virtual environment\n", "python -m venv my_virtual_env\n", "python -m venv --system-site-packages my_virtual_env\n", "\n", "# load, use and unload the virtual environment\n", "source my_virtual_env/bin/activate\n", "pip install sqlalchemy\n", "# install package from a custom repository (https://artifactory.fifty.eu)\n", "pip install --index-url=https://artifactory.fifty.eu/artifactory/api/pypi/pypi/simple/ odin-data-access\n", "deactivate\n", "\n", "# one can alternatively use the python \"wrapper\" of the virtual env\n", "my_virtual_env/bin/python -m pip install sqlalchemy\n", "```\n", "\n", "*--system-site-packages* will keep the original *site-packages* folders at the end of `sys.path`" ] }, { "cell_type": "markdown", "id": "299641e6-3d87-4a86-a124-eb49edb75b4b", "metadata": {}, "source": [ "## Creating and maintaining a Python environment (cont...)\n", "\n", "Poetry [https://python-poetry.org](https://python-poetry.org) is the prefered environment and dependency management tool at Statnett.\n", "\n", "```bash\n", "# create project and a virtual environment from scratch\n", "poetry new my-project\n", "\n", "# ... or use Poetry with an existing one\n", "cd my-project\n", "poetry init\n", "\n", "# edit pyproject.toml for your needs (f.i. add dependencies, metadata ... etc),\n", "# create virtual environment and install dependencies\n", "poetry install\n", "# it will create the file poetry.lock\n", "# finally commit your poetry.lock file to version control\n", "\n", "# update all dependencies and poetry.lock\n", "poetry update\n", "```\n", "\n", "For more info: [https://python-poetry.org/docs/basic-usage/](https://python-poetry.org/docs/basic-usage/)" ] }, { "cell_type": "markdown", "id": "a60540b1-ad17-4c40-88e1-e6d6af1ab219", "metadata": {}, "source": [ "## Mutables vs. immutables\n", "\n", "Immutable object is an object with a fixed value. Immutable objects include `bool`, `int`, `float`, `str`, `bytes` and `tuple`. Such an object cannot be altered. A new object has to be created if a different value has to be stored. They play an important role in places where a constant hash value is needed, for example as a key in a dictionary.\n", "\n", "All objects that are not immutable are... mutable. All *hashable objects* **should** be immutable or use `id()`." ] }, { "cell_type": "code", "execution_count": 53, "id": "f4a4ea99-4038-4b61-9b8d-9ce623e6a663", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "35082617072" ] }, "execution_count": 53, "metadata": {}, "output_type": "execute_result" }, { "data": { "text/plain": [ "35082617104" ] }, "execution_count": 53, "metadata": {}, "output_type": "execute_result" }, { "data": { "text/plain": [ "'H'" ] }, "execution_count": 53, "metadata": {}, "output_type": "execute_result" }, { "data": { "text/plain": [ "-6333845781340707986" ] }, "execution_count": 53, "metadata": {}, "output_type": "execute_result" }, { "data": { "text/plain": [ "36350585520" ] }, "execution_count": 53, "metadata": {}, "output_type": "execute_result" }, { "data": { "text/plain": [ "36350587520" ] }, "execution_count": 53, "metadata": {}, "output_type": "execute_result" }, { "data": { "text/plain": [ "8927089887582119463" ] }, "execution_count": 53, "metadata": {}, "output_type": "execute_result" }, { "data": { "text/plain": [ "8927089887582119463" ] }, "execution_count": 53, "metadata": {}, "output_type": "execute_result" } ], "source": [ "i = 1\n", "id(i) # returns f.i. 9788992\n", "i += 1 # same as i = i + 1\n", "id(i) # returns a different value, hence - a brand new object\n", "\n", "s = 'Hello'\n", "s += ' World' # s is now a different object\n", "s[0] # 'H'\n", "# s[0] = 'h' # TypeError: 'str' object does not support item assignment\n", "\n", "t = (1, 4) # tuple\n", "l = [1, 4] # list\n", "hash(t) # returns f.i. -6333845781340707986\n", "# hash(l) # TypeError: unhashable type: 'list'\n", "\n", "s = 'the long and winding road'\n", "s2 = 'the long and winding road'\n", "\n", "# check if s and s2 are the same object:\n", "id(s) # Out: 139858905258704\n", "id(s2) # Out: 139858926037472\n", "\n", "# the hash should be the same\n", "hash(s) # Out: 7030216208569256362\n", "hash(s2) # Out: 7030216208569256362" ] }, { "cell_type": "markdown", "id": "ba19a7d1-866c-40dc-ba9e-a04c8af5f1ba", "metadata": {}, "source": [ "## Functions\n", "\n", "A function is a sequence of program instructions that performs a specific task, packaged as a unit.\n", "\n", "\n", "### Why use functions?\n", "\n", "Functions let you:\n", "\n", "- reuse code across several programs / projects\n", "\n", "- minimize code duplication\n", "\n", "- devide larger programming tasks\n", "\n", "- hide implementation details\n", "\n", "- improve readability\n", "\n", "- improve traceability\n", "\n", "\n", "### Any downside?\n", "\n", "Function calls bring some overhead pushing / popping function-data into / from stack.\n", "\n", "```python\n", "empty_list = list()\n", "empty_dict = dict()\n", "\n", "# better since it avoids the function call:\n", "empty_list = []\n", "empty_dict = {}\n", "```\n", "\n", "\n", "### Definitions\n", "\n", "Important definitions (may have different meanings in different programming languages):\n", "\n", "- *parameter / formal parameter* - the names that appear in a function definition. Parameters define what kind of arguments a function can accept.\n", "\n", "- *argument / actual parameter* - the values actually passed to a function when calling it.\n", "\n", "\n", "The keyword `def` introduces a function definition.\n", "\n", "Given the function definition:\n", "\n", "```python\n", "def func(foo, bar=None, **kwargs):\n", " pass\n", "```\n", "\n", "`foo`, `bar` and `kwargs` are **parameters** of `func`. However, when calling func, for example:\n", "\n", "```python\n", "func(42, bar=314, extra=somevar)\n", "```\n", "\n", "the values `42`, `314`, and `somevar` are arguments." ] }, { "cell_type": "code", "execution_count": 54, "id": "bf95f44a-1257-440f-b35f-a8536c8c4363", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "7" ] }, "execution_count": 54, "metadata": {}, "output_type": "execute_result" }, { "name": "stdout", "output_type": "stream", "text": [ "9\n" ] }, { "data": { "text/plain": [ "[9, -2]" ] }, "execution_count": 54, "metadata": {}, "output_type": "execute_result" }, { "name": "stdout", "output_type": "stream", "text": [ "[9, 5]\n" ] } ], "source": [ "# Functions (example1)\n", "\n", "# case1: assuming that the arguments are of ummutable type (f.i. int):\n", "def add(a, b): # - function definition / header\n", " \"\"\"Function for adding two integers\"\"\" # - docstring\n", " # the function body will not be evaluated (executed) before the function is called\n", " result = a + b\n", " a = 5 # will not change the corresponding argument since it is immutable\n", " return result # - function that does not contain return, implicitly returns None\n", "\n", "# defining two integer variables to be used as arguments when calling the function `add`\n", "a_var = 9\n", "b_var = -2\n", "\n", "add(a_var, b_var) # Out: 7\n", "\n", "# even though a_var and a point at the same data (they are references: a = a_var),\n", "# integers are immutable and a_var will remain unchanged\n", "print(a_var) # Out: 9\n", "\n", "\n", "# case2: assuming that the arguments are of mutable type (f.i. list)\n", "def addl(a, b):\n", " \"\"\"Function for adding two lists\"\"\"\n", " # assuming that a and b are lists\n", " result = a + b # produces a new list that contains the elements of `a` followed by the elements of `b`\n", " a.append(5) # appending a new int element - 5 to the list `a`\n", " return result\n", "\n", "# defining two lists with one element each to be used as arguments when calling `addl`\n", "a_var = [9]\n", "b_var = [-2]\n", "\n", "addl(a_var, b_var) # Out: [9, -2]\n", "\n", "# since a (inside the function) is a reference of a_var (a = a_var) and\n", "# lists are mutable and `a_var` will be changed inside the function\n", "print(a_var) # Out: [9, 5]" ] }, { "cell_type": "code", "execution_count": 55, "id": "ecd03ee2-3523-40bd-afd1-4263a777bae0", "metadata": {}, "outputs": [], "source": [ "# Functions (example2) - parameters and arguments\n", "\n", "def add(a, b):\n", " \"\"\"Function for adding integers\"\"\"\n", " return a + b\n", "\n", "# positional arguments:\n", "# the order by which the arguments are sent to the function decides\n", "# which argument is assigned to which parameter\n", "my_result = add(2, 5)\n", "\n", "# keyword arguments:\n", "# which argument is assigned to which parameter is decided by\n", "# referrig directly to the argument names - keyword arguments (kwargs)\n", "my_result = add(b=5, a=2)\n", "\n", "my_tuple = (2, 5)\n", "my_dict = {'b': 5, 'a': 2}\n", "\n", "my_result = add(*my_tuple) # unpacked and assigned to the positional arguments\n", "my_result = add(**my_dict) # unpacked and assigned to the kw. arguments\n", "\n", "# defining the function `add`with defaukt value for argument `b`\n", "def add(a, b=5):\n", " \"\"\"Function for adding integers\"\"\"\n", " return a + b\n", "my_result = add(2) # the default value for `b` is used unless a parameter is sent / used\n", "\n", "# assigning a varying amount of arguments to parameters\n", "# # args - tuple containing all positional arguments (with the exception of `a`)\n", "# kwargs - a {keyword: value} dict containing all keyword arguments\n", "def add(a, *args, **kwargs):\n", " \"\"\"Function for adding integers\"\"\"\n", " if args:\n", " b = args[0]\n", " elif 'b' in kwargs:\n", " b = kwargs['b']\n", " return a + b\n", "\n", "my_result = add(2, 5, 9, 11) # 5 assigned to args[0]\n", "my_result = add(2, 6, 5, b=5, c=9, d=11) # 5 assigned to kwargs['b']\n", "\n", "# my_result = add(2, b=5, 9, 11) # SyntaxError: positional argument follows keyword argument" ] }, { "cell_type": "code", "execution_count": 56, "id": "c67ca162-e8eb-4b4d-8c3c-246b000c5c13", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "10 20 {'a': 1, 'b': 2, 'c': 3}\n" ] } ], "source": [ "# Positional only parameters:\n", "# def f(pos1, pos2, /, pos_or_kwd, *, kwd1, kwd2):\n", "# ----------- ---------- ----------\n", "# | | |\n", "# | Positional or keyword |\n", "# | - Keyword only\n", "# -- Positional only\n", "\n", "# Positional-only parameters give more control to library authors to better\n", "# express the intended usage of an API and allows the API to evolve in a safe, backward-compatible way.\n", "# Additionally, it makes the Python language more consistent with existing documentation and\n", "# the behavior of various “builtin” and standard library functions.\n", "\n", "# One use case for this notation is that it allows pure Python functions to fully emulate behaviors of existing C coded functions.\n", "# For example, the built-in divmod() function does not accept keyword arguments:\n", "def divmod(a, b, /):\n", " \"\"\"Emulate the built in divmod() function\"\"\"\n", " return (a // b, a % b)\n", "\n", "# Another use case is to preclude keyword arguments when the parameter name is not helpful.\n", "# For example, the builtin len() function has the signature len(obj, /).\n", "# This precludes awkward calls such as: len(obj='hello'), where the \"obj\" keyword argument impairs readability.\n", "\n", "# A further benefit of marking a parameter as positional-only is that it allows the parameter name to be changed in the future without risk of breaking client code.\n", "\n", "def my_func(a, b, /, **kwargs):\n", " print(a, b, kwargs)\n", "\n", "my_func(10, 20, a=1, b=2, c=3)" ] }, { "cell_type": "markdown", "id": "26a4f2fe-b79e-4e25-9107-230a53d11373", "metadata": {}, "source": [ "## Functions (cont ...)\n", "\n", "Docstrings annotations and other hints\n", "\n", "```python\n", "def decrypt(password: str, edata: str) -> str:\n", " \"\"\"\n", " Decrypts `edata` using `password`.\n", "\n", " `edata` is in the following format:\n", " enc-val$`version-num`$`bas64-salt`$`base64-encrypted_data`\n", "\n", " :param password: The password to generate the key with\n", " :type password: str\n", "\n", " :param edata: The data to be decrypted\n", " :type edata: str\n", "\n", " :raises EtoolkitInstanceError: If the encryption format is unsupported\n", "\n", " :return: The output string / decrypted data\n", " :rtype: str\n", " \"\"\"\n", " if not edata.startswith('enc-val$1$'):\n", " raise EtoolkitInstanceError('Unsupported encryption format')\n", " # some more code magic coming after....\n", " # ...\n", " # ..\n", " return decrypted_str\n", "```\n", "\n", "Using `typing` for more advanced / flexible hinting\n", "\n", "```python\n", " \n", "import typing\n", "\n", "Basestring = typing.Union[str, bytes]\n", "\n", "def decrypt(password: Basestring, edata: str) -> str:\n", " pass\n", "\n", "\n", "# or simply...\n", "from typing import Union\n", "\n", "def decrypt(password: Union[str, bytes], edata: str) -> str:\n", " \"\"\"Generic documentation. No need for pass\"\"\"\n", "\n", "# Python >= 3.10 only\n", "def decrypt(password: str | bytes, edata: str) -> str:\n", " \"\"\"Generic documentation. No need for pass\"\"\"\n", "```" ] }, { "cell_type": "markdown", "id": "40b749b3-5ec6-4c81-a9ee-3327433b4997", "metadata": {}, "source": [ "## Functions (cont ...)\n", "\n", "Functions as parameters / arguments, lambdas and returning multiple values\n", "\n", "Functions in Python are callable objects. Callable objects can be created by defining the `__call__` method. More on that later in the course..." ] }, { "cell_type": "code", "execution_count": 57, "id": "edf9cfa7-d76c-428b-a39f-8c0697789031", "metadata": {}, "outputs": [], "source": [ "# example: function as a parameter / argument\n", "# the built-in function map: map(function, iterable, *iterables)\n", "\n", "\n", "def fetch_the_first_letter(input_str: str) -> str:\n", " \"\"\"Fetches the first letter of the string input_str or 'x'\"\"\"\n", " try:\n", " return input_str[0]\n", " except Exception:\n", " return 'x'\n", "letter_list = list(map(fetch_the_first_letter, ['foo', 'bar', 'test'])) # Out: ['f', 'b', 't']\n", "\n", "\n", "# Small anonymous functions can be created with the lambda keyword\n", "letter_list = list(map(lambda x: x[0], ['foo', 'bar', 'test'])) # Out: ['f', 'b', 't']\n", "\n", "\n", "# example: function returning multiple values\n", "# A function can return multiple values by implicitly returning a tuple:\n", "def square_cube(x):\n", " \"\"\"returns x, x^2 and x^3\"\"\"\n", " return x, x**2, x**3\n", "\n", "numbers = square_cube(5) # Out: (5, 25, 125)\n", "num, sqnum, cbnum = square_cube(5) # unpacking the tuple" ] }, { "cell_type": "markdown", "id": "d890df54-4097-43b5-a96d-1d9996e5626f", "metadata": {}, "source": [ "## Functions (cont ...)\n", "\n", "Enclosing and nested functions\n", "\n", "Can be used as:\n", "\n", "- regular functions within functions\n", "\n", "- dynamic function factories" ] }, { "cell_type": "code", "execution_count": 58, "id": "ea583c17-c938-495a-9586-2027c66328a2", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "9\n", "15\n" ] } ], "source": [ "# enclosing functions\n", "# used when:\n", "# - we want to dynamically generate a function\n", "# - we want to hide implementation details (encapsulation)\n", "\n", "# when we want to dynamically generate a function:\n", "# the function will behave differently depending on what parameters were\n", "# used when calling its enclosing function (factory function)\n", "\n", "def get_multiplier_of(base: int) -> str:\n", " \"\"\"the function enclosing its nested functions\"\"\"\n", " # this function is the enclosing function of the function `multiplier_function`\n", "\n", " def multiplier_function(x):\n", " \"\"\"a nested function\"\"\"\n", " return base * x\n", "\n", " return multiplier_function\n", "\n", "times3 = get_multiplier_of(3)\n", "times5 = get_multiplier_of(5)\n", "print(times3(3)) # Out: 9\n", "print(times5(3)) # Out: 15" ] }, { "cell_type": "markdown", "id": "cb5d1ad1-b702-41c3-92c4-93ed3df4e0c0", "metadata": {}, "source": [ "## Scopes in Python\n", "\n", "- local - assigned names are local unless declared global\n", "\n", "- enclosed - the scope of the variable inside a function with a nested function\n", "\n", "- global - global for the current module\n", "\n", "- built-in\n", "\n", "`locals()` and `globals()` return dicts of symbols for their respective scopes" ] }, { "cell_type": "code", "execution_count": 59, "id": "49d7f6e1-1f54-4b96-8203-5eae95f47ebe", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "7 8\n", "100\n", "{'id': 200, 'num3': 100, 'num5': 23}\n", "{'num3': 100, 'num5': 23}\n", "100\n", "{'num3': 100, 'num5': 25}\n", "100\n", "25\n", "99\n" ] } ], "source": [ "# module globals\n", "num1 = 7\n", "num2 = 8\n", "\n", "def print_numbers():\n", " print(num1, num2) # module globals. Out: 7 8\n", " num3 = 100\n", " print(num3) # prints 100, num3 is in the function (local) scope\n", " global num4 # assignes / references num4 to / in the global scope\n", " num4 = 99\n", " id = 200 # new symbol in local scope\n", " # id(num4) # will not yield the expected result (raises TypeError)\n", " num5 = 23\n", " \n", " # locals() updates and then returns a dictionary representing the current local symbol table\n", " print(locals())\n", "\n", " def print_numbers2():\n", " print(locals()) # N.B. Updates and then displays the local symbol table (num3 will be included)\n", " print(num3) # OK, enclosed scope\n", " # num3 = 9 # UnboundLocalError: local variable 'num3' referenced before assignment\n", " nonlocal num5 # assignes / references num5 to / in enclosing scope. N.B. num5 must exist in `print_numbers`\n", " num5 = 25\n", " print(locals())\n", "\n", " print_numbers2() # Out: 100\n", " print(num3)\n", " print(num5) # Out: 25\n", "\n", "print_numbers()\n", "# print(num3) # Raises NameError - why?\n", "print(num4) # Out: 99\n", "# print_numbers2() # Raises NameError" ] }, { "cell_type": "markdown", "id": "5c91e152-a658-4811-832f-90712d708f8d", "metadata": {}, "source": [ "## Decorators\n", "\n", "Decorators can be used to modify the behavior of the objects they decorate. Decorators can be implemented either by using classes or by using nested functions.\n", "\n", "```python \n", "def my_decorator(func):\n", "\n", " def decorated():\n", " # we place the logic we want to take place before the decorated function's logic here\n", " print('Doing something before the decorated function')\n", " retval = func() # calling the decorated function and (optionally) taking care of its return value\n", " # we place the logic we want to take place after the decorated function's logic here\n", " print('Doing something after the decorated function')\n", " return retval # returning the return value of the original (decorated) function\n", " return decorated\n", "```\n", "\n", "once having a decorator:\n", "\n", "```python\n", "def my_function():\n", " print('Alice')\n", "\n", "my_function = my_decorator(my_function)\n", "my_function()\n", "``` \n", "\n", "... may be dificult to read / understand, while:\n", "\n", "```python \n", "@my_decorator\n", "def my_function():\n", " print('Alice')\n", "\n", "my_function()\n", "``` \n", "\n", "... may be easier" ] }, { "cell_type": "code", "execution_count": 60, "id": "4346260d-76af-437e-9c10-1c1d0c478695", "metadata": {}, "outputs": [], "source": [ "# Decorators (cont ...) - a complete example\n", "\n", "import sys\n", "from functools import wraps\n", "\n", "def requires_access(access_secret: str):\n", "\n", " def api_access_decorator(f):\n", "\n", " @wraps(f)\n", " def decorated(*args, **kwargs):\n", " if 'secret' not in kwargs:\n", " sys.exit('No secret provided')\n", " if not kwargs['secret'] or kwargs['secret'] != access_secret:\n", " sys.exit(\"Secret doesn't match\")\n", " # return f(args[0], **kwargs)\n", " return f(*args, **kwargs)\n", "\n", " return decorated\n", "\n", " return api_access_decorator\n", "\n", "\n", "@requires_access(access_secret='b28cfeaa65b73cf')\n", "@is_admin\n", "def sensitive_function(data, **kwargs):\n", " \"\"\"very sensitive function\"\"\"\n", " db.save(data)" ] }, { "cell_type": "markdown", "id": "14ea817a-0748-4941-8899-0b9025f25a93", "metadata": {}, "source": [ "## String formatting\n", "\n", "The old ways...\n", "\n", "```python \n", "f = 6.57865\n", "i = 27\n", "s = 'another string'\n", "\n", "\n", "'%s - %d - %5.2f' % (s, i, f) # Out: 'another string - 27 - 6.58'\n", "\n", "# still used in:\n", "logger.debug(\"%d - %s\", event.id, message)\n", "\n", "# using the .format method\n", "'{} - {} - {:5.2f}'.format(s, i, f) # implicit\n", "'{0} - {1} - {2:5.2f}'.format(s, i, f) # explicit\n", "'{my_str} - {i} - {fl:5.2f}'.format(my_str=s, fl=f, i=i) # keyword\n", "# Out: 'another string - 27 - 6.58'\n", "\n", "\n", "# modern Python >= 3.6 f-strings\n", "f'{s} - {i} - {f:5.2f}' # Out: 'another string - 27 - 6.58'\n", "``` \n", "\n", "See https://docs.python.org/3/library/string.html#formatspec for the complete format specification" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.1" } }, "nbformat": 4, "nbformat_minor": 5 }