1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
|
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Introduction to Python</title>
<meta name="author" content="Simeon Simeonov">
<meta name="apple-mobile-web-app-capable" content="yes">
<meta name="apple-mobile-web-app-status-bar-style" content="black-translucent">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link rel="stylesheet" href="dist/reset.css">
<link rel="stylesheet" href="dist/reveal.css">
<link rel="stylesheet" href="dist/theme/statnett.css" id="theme">
<!-- Theme used for syntax highlighting of code -->
<link rel="stylesheet" href="plugin/highlight/monokai.css" id="highlight-theme">
<!-- <link rel="stylesheet" href="plugin/highlight/zenburn.css" id="highlight-theme"> -->
</head>
<body>
<div class="reveal">
<!-- Any section element inside of this container is displayed as a slide -->
<div class="slides">
<section>
<h2>Introduction to Python</h2>
<h4>Data Engineering @ Statnett</h4>
</br>
<p><small>Simeon Simeonov</small></p>
</section>
<section>
<h2>Goals</h2>
</br>
<ul>
<li><p>Present the Python programming language in a different way than <a href="https://docs.python.org">https://docs.python.org</a></p></li>
<li><p>Avoid information overload</p></li>
<li><p>Use examples and interaction rather than documents and slides</p></li>
</ul>
</section>
<section>
<h2>Preliminary plan</h2>
</br>
<ul>
<li><p>Basics: About the language, the Python eco-system, types, scopes, modules and functions</p></li>
<li><p>Control flow: if / for / while / try, iterators, decorators, "tactical programming" tips</p></li>
<li><p>Object orientation: How Python "really works"</p></li>
<li><p>A brief tour through Python's standard library</p></li>
<li><p>Code design and best practices: How to design your code</p></li>
</ul>
</section>
<section>
<h2>What is Python?</h2>
</br>
<ul>
<li><p>Python is an interpreted high-level general-purpose programming language - advanced through the Python Enhancement Proposal (PEP) process</p></li>
<li><p>CPython is the reference implementation of Python, written in C (alternatives: pypy, jython)</p></li>
<li><p>python - interpreter and interpreter shell (alternatives: ipython, bpython)</p></li>
<li><p>libpython</p></li>
<li><p>Calling C from Python: Cython, CFFI, ctypes</p></li>
</ul>
</section>
<section>
<h2>Philosophy</h2>
</br>
<pre data-id="code-animation">
<code class="bash" data-trim type="text/template">
$ python
</code>
</pre>
<pre data-id="code-animation">
<code class="python" data-trim type="text/template">
import this
# The Zen of Python, by Tim Peters
# Beautiful is better than ugly.
# Explicit is better than implicit.
# Simple is better than complex.
# Complex is better than complicated.
# Flat is better than nested.
# Sparse is better than dense.
# Readability counts.
# Special cases aren't special enough to break the rules.
# Although practicality beats purity.
# Errors should never pass silently.
# Unless explicitly silenced.
# In the face of ambiguity, refuse the temptation to guess.
# There should be one-- and preferably only one --obvious way to do it.
# Although that way may not be obvious at first unless you're Dutch.
# Now is better than never.
# Although never is often better than *right* now.
# If the implementation is hard to explain, it's a bad idea.
# If the implementation is easy to explain, it may be a good idea.
# Namespaces are one honking great idea -- let's do more of those!
</code>
</pre>
</section>
<section>
<h2>Built-in functions</h2>
</br>
<p>Few built-in functions.</p>
<p><a href="https://docs.python.org/3/library/functions.html">https://docs.python.org/3/library/functions.html</a></p>
<ul>
<li><em>dir([obj])</em> - returns a list of valid attributes for that object</li>
<li><em>id(obj)</em> - returns the "identity" of an object - an integer which is guaranteed to be unique</li>
<li><em>print(...)</em> - prints objects to a text stream</li>
<li><em>str(...)</em> - returns a string version of object</li>
<li><em>type(obj)</em> - returns the type of an object</li>
</ul>
</section>
<section>
<h2>Common built-in types</h2>
</br>
<p>Python uses <a href="https://en.wikipedia.org/wiki/Duck_typing"><em>duck typing</em></a> and has typed objects but untyped variable names.</p>
<p>Type constraints are not checked at compile time; rather, operations on an object may fail, signifying that the given object is not of a suitable type. Despite being dynamically-typed, Python is strongly-typed, forbidding operations that are not well-defined (for example, adding a number to a string) rather than silently attempting to make sense of them.</p>
<pre data-id="code-animation">
<code class="python" data-trim type="text/template">
s = 'foo' # this is a string / str, same as str('foo'), may be encoded, immutable
b = b'foo' # bytes, same as bytes('foo', 'utf-8'), may be decoded, immutable
i = 6 # int, same as int('6'), immutable
f = 0.1 # float, same as float('0.1'), immutable
b = False # bool, same as bool(0), bool(''), bool(None)... immutable / constant
n = None # NoneType, similar to 'null' in other languages, immutable / constant
l = [1, False, 'foo'] # list, same as list((1, False, 'foo'))
t = (1, False, 'foo') # tuple, same as tuple([1, False, 'foo']), immutable
d = {'foo': 1, 'bar': 8} # dict, same as dict(foo=1, bar=8), similar to hash in other languages
s = {'foo', 'bar', 1, 1, 4} # set, same as set(['foo', 'bar', 1, 1, 4]), removes duplicates
</code>
</pre>
<p>Classes, functions, modules, .... and even types are simply other types :)</p>
</section>
<section>
<h2>Modules</h2>
</br>
<p>A module is a file containing Python definitions and statements. The file name is the module name with the suffix <em>.py</em> appended. Within a module, the module's name (as a string) is available as the value of the global variable <em>__name__</em></p>
<p>When a module named <em>foo</em> is imported, the interpreter first searches for a built-in module with that name (<em>sys.builtin_module_names</em>). If not found, it then searches for a file named <em>foo.py</em> in a list of directories given by the variable <em>sys.path</em>. <em>sys.path</em> is initialized from these locations:</p>
<ul>
<li>the directory containing the input script (or the current directory when no file is specified)</li>
<li><em>PYTHONPATH</em> - env. variable - a list of directory names</li>
<li>the installation-dependent default locations</li>
</ul>
<p>The module is then imported only once and "cached" in <em>sys.modules</em></p>
</section>
<section>
<h2>Packages</h2>
</br>
<p><a href="https://docs.python.org/3/tutorial/modules.html#packages">Packages</a> are a way of structuring Python's module namespace by using "dotted module names"</p>
<p>The import statement combines two operations:</p>
<ul>
<li>it searches for the named module</li>
<li>it binds the results of that search to a name in the <em>local scope</em></li>
</ul>
<pre data-id="code-animation">
<code class="python" data-trim type="text/template">
# bar.py then bar/__init__.py will be considered, the first match executed and bound to 'bar'
import bar
import mymodule.foo # implicitly executes mymodule.py, mymodule/__init__.py and mymodule/foo/__init__.py
import numpy as np # will be bound as 'np' instead of 'numpy'. N.B. __name__ is still 'numpy'
import some.extremely.deep.path.Animal as Animal # "sacrifice" the namespace in the name of convinience
from sys import path # execute sys and only import the 'path' attribute into local scope as 'path'
# relative imports must be explicit in Python 3
from .othermodule import something # expects that current module and 'othermodule' are in the same
# package (containing __init__.py)
from sys import * # NO! Bad programming practice since 1879
</code>
</pre>
</section>
<section>
<h2>Creating and maintaining a Python environment</h2>
</br>
<p>Python's official package repository is <a href="https://pypi.org"><em>PyPi (https://pypi.org)</em></a>, while Python's official package installer is <a href="https://pypi.org/project/pip/"><em>pip (https://pypi.org/project/pip/)</em></a></p>
<p>A Python environment is the physical and logical arrangement of Python modules and packages. Several options exist:</p>
<ul>
<li>using a proper operating system :) (symlinks, real commercial support etc.)</li>
<li>using <em>venv</em></li>
<li>using higher level tools like <em>poetry</em></li>
<li>using a mixture / cocktail of all of the above :)</li>
</ul>
</section>
<section>
<h2>Creating and maintaining a Python environment (cont...)</h2>
</br>
<p>Desired qualities for a flexible Python environment:</p>
<ul>
<li>easy to create and (un)load</li>
<li>do not require extra privileges</li>
<li>don't repeat yourself (DRY)</li>
<li>easy to update without breaking the API</li>
<li>easy to debug</li>
<li>play nicely with the VCS (git)</li>
</ul>
</section>
<section>
<h3>Creating and maintaining a Python environment (cont...)</h3>
</br>
<p>Exploting the operating system can be done by:</p>
<ul>
<li>(re)defining <em>PYTHONPATH</em></li>
<li>using symlinks to point at packages placed at different locations</li>
</section>
<section>
<h3>Creating and maintaining a Python environment (cont...)</h3>
</br>
<p>Using <em>venv</em> can be done by directly invoking <em>python</em>:</p>
<pre data-id="code-animation">
<code class="bash" data-trim type="text/template">
# create a virtual environment
python -m venv my_virtual_env
python -m venv --system-site-packages my_virtual_env
# load, use and unload the virtual environment
source my_virtual_env/bin/activate
pip install sqlalchemy
deactivate
# one can alternatively use the python "wrapper" of the virtual env
my_virtual_env/bin/python -m pip install sqlalchemy
</code>
</pre>
<p><em>--system-site-packages</em> will keep the original <em>site-packages</em> folders at the end of <em>sys.path</em></p>
</section>
<section>
<h3>Creating and maintaining a Python environment (cont...)</h3>
</br>
<p><em>Poetry</em> (<a href="https://python-poetry.org"><em>https://python-poetry.org</em></a>) is the prefered environment and dependency management tool at Statnett.</p>
<pre data-id="code-animation">
<code class="bash" data-trim type="text/template">
# create project and a virtual environment from scratch
poetry new my-project
# ... or use Poetry with an existing one
cd my-project
poetry init
# edit pyproject.toml for your needs (f.i. add dependencies, metadata ... etc),
# create virtual environment and install dependencies
poetry install
# finally commit your poetry.lock file to version control
# update all dependencies
poetry update
</code>
</pre>
<p>For more info: <a href="https://python-poetry.org/docs/basic-usage/"><em>https://python-poetry.org/docs/basic-usage/</em></a></p>
</section>
<section>
<h2>Mutables vs. immutables</h2>
<p>Immutable object is an object with a fixed value. Immutable objects include <em>bool, int, float, str, bytes</em> and <em>tuples</em>. Such an object cannot be altered. A new object has to be created if a different value has to be stored. They play an important role in places where a constant hash value is needed, for example as a key in a dictionary.</p>
<p>All objects that are not immutable are... mutable. All <em>hashable objects</em> <b>should</b> be immutable or use <em>id()</em>.</p>
<pre data-id="code-animation">
<code class="python" data-trim type="text/template">
i = 1
id(i) # returns f.i. 9788992
i += 1 # same as i = i + 1
id(i) # returns a different value, hence - a brand new object
s = 'Hello'
s += ' World' # s is now a different object
s[0] # 'H'
s[0] = 'h' # TypeError: 'str' object does not support item assignment
t = (1, 4) # tuple
l = [1, 4] # list
hash(t) # returns f.i. -6333845781340707986
hash(l) # TypeError: unhashable type: 'list'
s = 'the long and winding road'
s2 = 'the long and winding road'
# check if s and s2 are the same object:
id(s) # Out: 139858905258704
id(s2) # Out: 139858926037472
# the hash should be the same
hash(s) # Out: 7030216208569256362
hash(s2) # Out: 7030216208569256362
</code>
</pre>
</section>
<section>
<h2>Functions</h2>
</br>
<p>A function is a sequence of program instructions that performs a specific task, packaged as a unit.</p>
<p>Functions let you:</p>
<ul>
<li>reuse code across several programs / projects</li>
<li>minimize code duplication</li>
<li>devide larger programming tasks</li>
<li>hide implementation details</li>
<li>improve readability</li>
<li>improve traceability</li>
</ul>
<p>Function calls bring some overhead pushing / popping function-data into / from stack.</p>
<p>Important definitions (may have different meanings in different programming languages):</p>
<ul>
<li><em>parameter / formal parameter</em> - variable / data provided as input to the function</li>
<li><em>argument / actual parameter</em> - local variable / data to the given function</li>
</ul>
<p>The keyword <em>def</em> introduces a function definition.</p>
</section>
<section>
<h2>Functions (cont...)</h2>
</br>
<pre data-id="code-animation">
<code class="python" data-trim type="text/template">
def add(a, b): # - function definition / header
"""Function for adding integers""" # - docstring
result = a + b
a = 5
return result # - function that does not contain return, implicitly returns None
a_param = 9
b_param = -2
add(a_param, b_param) # Out: 7
# integers are immutable and a_param will remain unchanged
print(a_param) # Out: 9
def addl(a, b):
"""Function for adding two lists"""
result = a + b
a += [5]
return result
a_param = [9]
b_param = [-2]
addl(a_param, b_param) # Out: [9, -2]
# lists are mutable and a_param will be changed
print(a_param) # Out: [5]
</code>
</pre>
</section>
<section>
<h2>Functions</h2>
</br>
<h3>To be continued in the next episode....</h3>
</section>
<section>
<h2>Scopes in Python</h2>
</br>
<ul>
<li><em>local</em> - assigned names are local unless declared <em>global</em></li>
<li><em>enclosed</em> - the scope of the variable inside a function with a nested function</li>
<li><em>global - global for the current module</em></li>
<li><em>built-in</em></li>
</ul>
<p><em>locals()</em> and <em>globals()</em> return dicts of symbols for their respective scopes</p>
<pre data-id="code-animation">
<code class="python" data-trim type="text/template">
num1, num2 = 7, 8 # module globals
def print_numbers():
print(num1, num2) # OK, these are module globals
num3 = 100
print(num3) # prints 100, num3 is in the function (local) scope
global num4 # assignes / references num4 to / in the global scope
num4 = 99
id = 200 # new symbol in local scope
# id(num4) # will not yield the expected result (raises TypeError)
def print_numbers2():
print(num3) # OK, enclosed scope
print_numbers2() # prints 100
print_numbers()
# print(num3) # Raises NameError - why?
print(num4) # Prints 99 - why?
# print_numbers2() # Raises NameError
</code>
</pre>
</section>
<section>
<h1>Q & A</h1>
</section>
</div>
</div>
<script src="dist/reveal.js"></script>
<script src="plugin/zoom/zoom.js"></script>
<script src="plugin/notes/notes.js"></script>
<script src="plugin/search/search.js"></script>
<script src="plugin/markdown/markdown.js"></script>
<script src="plugin/highlight/highlight.js"></script>
<script>
// Also available as an ES module, see:
// https://revealjs.netlify.app/initialization/
Reveal.initialize({
controls: true,
progress: true,
center: true,
hash: true,
// Learn about plugins: https://revealjs.netlify.app/plugins/
plugins: [ RevealZoom, RevealNotes, RevealSearch, RevealMarkdown, RevealHighlight ]
});
Reveal.configure({ pdfSeparateFragments: false });
</script>
</body>
</html>
|
