Cover image of Python Bytes
(122)

Rank #103 in Technology category

Technology

Python Bytes

Updated 4 days ago

Rank #103 in Technology category

Technology
Read more

Python Bytes is a weekly podcast hosted by Michael Kennedy and Brian Okken. The show is a short discussion on the headlines and noteworthy news in the Python, developer, and data science space.

Read more

Python Bytes is a weekly podcast hosted by Michael Kennedy and Brian Okken. The show is a short discussion on the headlines and noteworthy news in the Python, developer, and data science space.

iTunes Ratings

122 Ratings
Average Ratings
111
4
3
2
2

Outstanding Podcast

By Aggienaut - Sep 21 2019
Read more
Always informative and entertaining!

Great Python info

By alltheothernamesweretaken54136 - Dec 18 2018
Read more
Great way to keep up with the goings-on in Python.

iTunes Ratings

122 Ratings
Average Ratings
111
4
3
2
2

Outstanding Podcast

By Aggienaut - Sep 21 2019
Read more
Always informative and entertaining!

Great Python info

By alltheothernamesweretaken54136 - Dec 18 2018
Read more
Great way to keep up with the goings-on in Python.

Listen to:

Cover image of Python Bytes

Python Bytes

Updated 4 days ago

Read more

Python Bytes is a weekly podcast hosted by Michael Kennedy and Brian Okken. The show is a short discussion on the headlines and noteworthy news in the Python, developer, and data science space.

#4 Python 3 is just fine for beginners thank you, q is awesome for debugging, and more

Podcast cover
Read more

This is Python Bytes, Python headlines and news deliver directly to your earbuds: episode 4: recorded on November 28, 2016. In this episode we cover the case for Python 3, asyncio, pyston, pydoc.io, and q.

This episode was brought to you by Rollbar: they help you take the pain out of errors.

News items

The Case FOR Python 3

>> "It's as simple as that. If you learn Python 2, then you can still work with all the legacy Python 2 code in existence **until Python dies or you (hopefully) move on**. But if you learn Python 3 then your future is very uncertain. You could really be learning a dead language and end up having to learn Python 2 anyway." -- Zed S.
  • This is harmful to the Python community.
  • Let's stop recommending this book and stop elevating this guy's work
  • "Arguments" from Zed:
    • Not In Your Best Interests: "The Python project’s efforts to convince you to start with Python 3 are not in your best interest, but, rather, are only in the best interests of the Python project."
    • You Should Be Able to Run 2 and 3 (in the same process). The fact that no one has (yet) written a Python 3 interpreter that can simultaneously run Python 2 in the same process means Python 3 is not Turing complete
    • Difficult To Use Strings: "The strings in Python 3 are very difficult to use for beginners. In an attempt to make their strings more “international” they turned them into difficult to use types with poor error messages."
    • Final analysis:
      • You can’t add b"hello" to "hello".
      • Too many formatting choices for strings
      • Bytes aren't automatically decoded to strings
    • Therefore: All newbies should avoid Python 3 like the plague - it will curse your career and cloud your judgement.
  • Excellent rebuttal article: Rebuttal for Python 3

Reddit discussion of a Feb article on "AsyncIO for the Working Python Developer"

  • Article is a nice introduction, but uses Python 3.4 syntax
  • Interesting to note that no one in the reddit comments suggests a better 3.5 tutorial. Is async/await just too confusing for a 3.5 intro article? Or is it so simple that no one has thought it needs a tutorial.
  • 23 page article / tutorial on Python 3.4's asyncio
  • Not covering Python 3.5's async / await, but great gateway article
  • Here’s a call for action. Someone write a similar article but with 3.5+ syntax. Also could we show examples without sleep calls. Sleep is such a lame way to show “do some actual work”. Maybe use a producer/consumer or reader/writer example or something.

Pyston 0.6 released

  • Pyston is an open source Python implementation that aims to be both highly compatible and high-performance.
  • It uses modern JIT techniques and natively supports many CPython C extension modules.
  • Pyston is sponsored by Dropbox, and is pronounced "piston".
  • main goal was to reduce the overall memory footprint. It also contains a lot of additional smaller changes for improving compatibility and fixing bugs.
    • We also spent time on making it easier to replace CPython with Pyston, such as by more closely matching its directory structure
    • following its ‘dict’ ordering.
    • We can now, for example, run pip and virtualenv unmodified, without requiring any upstream patches like other implementations do.
  • Unclear how much of a research project vs an actual project this is at the moment

Announcing pydoc.io

  • Automatic API documentation from pypi repositories.
  • Pydoc.io will eventually auto-generate API reference documentation for every package on PyPI.

What one thing took your Python to the next level

  1. Mastering generators
  2. Understanding how iteration really works in Python opens up many possibilities for elegant, high performance code. Use iPython. Learn that for does tuple unpacking; play with zip, enumerate, all, any; take a look at the itertools module
  3. Unit tests! Pytest in particular! Gives you a whole new perspective on your code and dammit, it's so satisfying to get all those green OKs!
  4. list comprehensions
  5. For me, all the work of David Beazley, in particular http://www.dabeaz.com/coroutines/

q : Quick-and-dirty debugging output for tired programmers

Our personal news

Michael

Brian

Nov 29 2016

21mins

Play

#3 Python 3.6 is coming, and it's awesome plus superior text processing with Pynini

Podcast cover
Read more

Welcome to Python Bytes. Python headlines delivered directly to your earbuds. In this episode we cover the new features in Python 3.6, text processing with Pynini, Python is 2nd most popular language on GitHub and more.

This episode was brought to you by Rollbar: they help you take the pain out of errors.

News items

How to get superior text processing with Pynini

  • how to change your thinking from regular expression matching to FSAs (finite-state acceptor), then FSTs.
  • Example using cheese names (love the Python tradition with that).
  • library developed at Google called pynini for easily using FSTs

Python 3.6 b4 Is out! (beta!!!)

  • PEP 498, formatted string literals.
  • PEP 515, underscores in numeric literals.
  • PEP 525, asynchronous generators.
  • PEP 530: asynchronous comprehensions.
  • The dict type has been reimplemented to use a faster, more compact representation similar to the PyPy dict implementation. This resulted in dictionaries using 20% to 25% less memory when compared to Python 3.5.
  • The class attribute definition order is now preserved.
  • Security: The new secrets module has been added to simplify the generation of cryptographically strong pseudo-random numbers suitable for managing secrets such as account authentication, tokens, and similar.
  • Windows: python.exe and pythonw.exe have been marked as long-path aware, which means that when the 260 character path limit may no longer apply. See removing the MAX_PATH limitation for details.

Create an Excellent Python Dev Env

  • pyenv, like virtualenv, I think, but independent of Python.
  • pyenv-virtualenv instead of virtualenvwrapper
  • pipsi - a tool to let you install python based CLI utilities in their own virtualenv easily.
  • Tools he installs with pipsi:
    • tox - testing multiple environments
    • mkdocs - documentation static site generator
    • git-review - gerrit integration with git.
    • flake8 - static analysis
  • He also mentions a shell called fish, which I hadn’t heard of before.

GitHub language statistics, Python is 2nd most popular language

  • An active repository should meet this requirements:
    created or updated during the evaluated period,
    • at least one star
    • at least one fork
    • a size larger than 10 Kb
  • JavaScript is likely over counted

Handling Unicode Strings in Python

  • Text Representation in Python and the differences between 2.7 and 3.4+
  • Converting between unicode strings and bytes
  • IO boundary issues with databases, file system, or network services
  • Logging
  • JSON encoding
  • Redis
  • A pointer to a 2012 article from Ned Batchelder called Pragmatic Unicode.

Python extensions for VS Code for Mac / PC / Linux

  • Linting (Prospector, Pylint, pycodestyle/Pep8, Flake8, pylama, pydocstyle with config files and plugins)
  • Intellisense (autocompletion with support for PEP-0484)
  • Scientific tools (Jupyter/IPython)
  • Auto indenting
  • Code formatting (autopep8, yapf, with config files)
  • Code refactoring (Rename, Extract Variable, Extract Method, Sort Imports)
  • Viewing references, code navigation, view signature
  • Excellent debugging support (remote debugging over SSH, mutliple threads, django, flask)
  • Unit testing, including debugging (unittest, pytest, nosetests, with config files)
  • Execute file or code in a python terminal
  • Local help file (offline documentation)
  • Snippets

Our personal news

Brian

  • Lot’s of writing. My target is to get the book in print by PyCon US 2017 in May. But that’s an aggressive schedule with the rest of my projects.
  • Still working on getting #25 out the door. Interview with Dave Hunt from Mozilla about Selenium, Mozilla, pytest, tox, CI, and much more. Should go out within the next couple of days.
  • Next interview scheduled is with David Hussman from DevJam. This will be a higher level discussion about software development practices. Look for that this month also.

Michael

Nov 23 2016

29mins

Play

#117 Is this the end of Python virtual environments?

Podcast cover
Read more

Sponsored by pythonbytes.fm/datadog

Brian #1: Goodbye Virtual Environments?

  • by Chad Smith
  • venv’s are great but they introduce some problems as well:
    • Learning curve: explaining “virtual environments” to people who just want to jump in and code is not always easy
    • Terminal isolation: Virtual Environments are activated and deactivated on a per-terminal basis
    • Cognitive overhead: Setting up, remembering installation location, activating/deactivating
  • PEP 582 — Python local packages directory
    • This PEP proposes to add to Python a mechanism to automatically recognize a __pypackages__directory and prefer importing packages installed in this location over user or global site-packages. This will avoid the steps to create, activate or deactivate “virtual environments”. Python will use the __pypackages__ from the base directory of the script when present.
  • Try it now with pythonloc
    • pythonloc is a drop in replacement for python and pip that automatically recognizes a __pypackages__ directory and prefers importing packages installed in this location over user or global site-packages. If you are familiar with node, __pypackages__ works similarly to node_modules.
    • Instead of running python you run pythonloc and the __pypackages__ path will automatically be searched first for packages. And instead of running pip you run piploc and it will install/uninstall from __pypackages__.

Michael #2: webassets

  • Bundles and minifies CSS & JS files
  • Been doing a lot of work to rank higher on the sites
  • That lead me to Google’s Lighthouse
  • Despite 25ms response time to the network, Google thought my site was “kinda slow”, yikes!
  • webassets has integration for the big three: Django, Flask, & Pyramid.
    • But I prefer to just generate them and serve them off disk
def build_asset(env: webassets.Environment,
files: List[str],
filters: str,
output: str):
bundle = webassets.Bundle(
*files,
filters=filters,
output=output,
env=env
)
bundle.build(force=True)

Brian #3: Bernat on Python Packaging

Michael #4: What the mock? — A cheatsheet for mocking in Python

  • Nice introduction
  • Some examples
@mock.patch('work.os')
def test_using_decorator(self, mocked_os):
work_on()
mocked_os.getcwd.assert_called_once()

And

def test_using_context_manager(self):
with mock.patch('work.os') as mocked_os:
work_on()
mocked_os.getcwd.assert_called_once()

Brian #5: Transitions: The easiest way to improve your tech talk

  • By Saron Yitbarek
  • Jeff Atwood of CodingHorror noted “The people who can write and communicate effectively are, all too often, the only people who get heard. They get to set the terms of the debate.”
  • Effectively presenting is part of effective communication.
  • I love the focus of this article. Focused on one little aspect of improving the performance of a tech talk.

Michael #6: Steering council announced

  • Our new leaders are
    • Barry Warsaw
    • Brett Cannon
    • Carol Willing
    • Guido van Rossum
    • Nick Coghlan
  • Via Joe Carey
  • We both think it’s great Guido is on the council.

From the list from Ant, my votes.

  • Q: What's the second movie about a database engineer called?
    A: The SQL.

  • !false
    It's funny 'cause it's true.

  • A programmer's spouse tells them, "Run to the store and pick up a loaf of bread. If they have eggs, get a dozen."
    The programmer comes home with 12 loaves of bread.

Feb 14 2019

28mins

Play

#77 You Don't Have To Be a Workaholic To Win

Podcast cover
Read more

Sponsored by Datadog: pythonbytes.fm/datadog

Brian #1: Why Senior Devs Write Dumb Code

  • Any fool can write code that a computer can understand. Good programmers write code that humans can understand.” - Kent Beck
  • Code that is clean, straightforward, obvious, and easy to read actually takes practice to achieve.
  • Follow principles like YAGNI, Singe Responsibility, DRY, etc.
  • Avoid clever one-liners, weird abstractions.
  • Esoteric language features.
  • Code needs to be readable and easily understood while under time and stress pressure.

Michael #2: GeoAlchemy 2

  • GeoAlchemy 2 provides extensions to SQLAlchemy for working with spatial databases.
  • GeoAlchemy 2 focuses on PostGIS. Aims to be simpler than its predecessor, GeoAlchemy.
  • Using it:
    • Connect (e.g. Postgres)
    • Declare a Mapping
class Lake(Base):
__tablename__ = 'lake'
id = Column(Integer, primary_key=True)
name = Column(String)
geom = Column(Geometry('POLYGON'))
  • Create a table (via the engine)
  • Create an Instance of the Mapped Class
  • Inserts like standard SQLAlchmey
  • Spatial Query
from sqlalchemy import func
query = session.query(Lake).filter(
func.ST_Contains(Lake.geom, 'POINT(4 1)'))

query = session.query(Lake.name,
Lake.geom.ST_Buffer(2).ST_Area().label('bufferarea'))

Brian #3: QtPyConvert

  • An automatic Python Qt binding transpiler to the Qt.py abstraction layer.
  • QtPyConvert supports the following bindings out of the box:
  • Conversions leave code comments in place, with the help of RedBaron
  • Converts to Qt.py
    • Minimal Python 2 & 3 shim around all Qt bindings - PySide, PySide2, PyQt4 and PyQt5

Michael #4: You Don't Have To Be a Workaholic To Win: 13 Alternative Ways To Stand Out

  • Do we have to kill ourselves to get ahead?
  • Don’t busy-brag
  • Max Q analogy
  • The tips
  • Creativity
  • Stubbornness
  • Curiosity
  • Kindness
  • Planning
  • Improvisation
  • Enthusiasm
  • Communication
  • Presence
  • Collaboration
  • Willingness
  • Patience
  • Institutional Knowledge

Brian #5: RedBaron

  • RedBaron is a python library to make the process of writing code that modify source code as easy and as simple as possible.
    • writing custom refactoring, generic refactoring, tools,
  • Used by QtPyConvert to achieve the conversion while leaving code comments in place
  • Uses the full syntax tree, FST. Like an AST, but keeps all information, including comments and formatting.
  • possible uses:
    • rename a variable in a source file... without clashing with things that are not a variable (example: stuff inside a string)
    • inline a function/method
    • extract a function/method from a series of line of code
    • split a class into several classes
    • split a file into several modules
    • convert your whole code base from one ORM to another
    • do custom refactoring operation not implemented by IDE

Michael #6: Project Beeware AppStore

  • Project BeeWare has just released its first iPhone app made in Python using its Briefcase tool.
  • Simple travel app for currency and tip calculations
  • Briefcase: A distutils extension to assist in packaging Python projects as standalone applications.
    Briefcase is a tool for converting a Python project into a standalone native application. You can package projects for:
    • Mac
    • Windows
    • Linux
    • iPhone/iPad
    • Android
    • AppleTV
    • tvOS
  • While there are other Python GUI toolkits aiming to enable Python developers to build and deploy iOS apps, like for instance the very nice Pythonista app, the BeeWare project is a bit different because it aims at cross-platform compatibility and native widgets with a set of different tools, like Briefcase and Toga.

Extras:

May 12 2018

21mins

Play

#95 Unleash the py-spy!

Podcast cover
Read more

Sponsored by DataDog -- pythonbytes.fm/datadog

Brian #1: dataset: databases for lazy people

  • dataset provides a simple abstraction layer removes most direct SQL statements without the necessity for a full ORM model - essentially, databases can be used like a JSON file or NoSQL store.
  • A simple data loading script using dataset might look like this:
import dataset

db = dataset.connect('sqlite:///:memory:')

table = db['sometable']
table.insert(dict(name='John Doe', age=37))
table.insert(dict(name='Jane Doe', age=34, gender='female'))

john = table.find_one(name='John Doe')

Michael #2: CuPy GPU NumPy

  • A NumPy-compatible matrix library accelerated by CUDA
  • How many cores does a modern GPU have?
  • CuPy's interface is highly compatible with NumPy; in most cases it can be used as a drop-in replacement.
  • You can easily make a custom CUDA kernel if you want to make your code run faster, requiring only a small code snippet of C++. CuPy automatically wraps and compiles it to make a CUDA binary
  • PyCon 2018 presentation: Shohei Hido - CuPy: A NumPy-compatible Library for GPU
  • Code example
>>> # This will run on your GPU!
>>> import cupy as np # This is the only non-NumPy line

>>> x = np.arange(6).reshape(2, 3).astype('f')
>>> x
array([[ 0., 1., 2.],
[ 3., 4., 5.]], dtype=float32)
>>> x.sum(axis=1)
array([ 3., 12.], dtype=float32)

Brian #3: Automate Python workflow using pre-commits

  • We covered pre-commit in episode 84, but I still had trouble getting my head around it.
  • This article by LJ Miranda does a great job with the workflow introduction and configuration necessary to get pre-commit working for black and flake8.
  • Includes a nice visual of the flow.
  • Demo of it all in action with a short video.

Michael #4: py-spy

  • Sampling profiler for Python programs
  • Written by Ben Frederickson
  • Lets you visualize what your Python program is spending time on without restarting the program or modifying the code in any way.
  • Written in Rust for speed
  • Doesn't run in the same process as the profiled Python program
  • Does NOT it interrupt the running program in any way.
  • This means Py-Spy is safe to use against production Python code.
  • The default visualization is a top-like live view of your python program
  • How does py-spy work? Py-spy works by directly reading the memory of the python program using the process_vm_readv system call on Linux, the vm_read call on OSX or the ReadProcessMemory call on Windows.

Brian #5: SymPy is a Python library for symbolic mathematics

  • “Symbolic computation deals with the computation of mathematical objects symbolically. This means that the mathematical objects are represented exactly, not approximately, and mathematical expressions with unevaluated variables are left in symbolic form.”
  • example:
>>> integrate(sin(x**2), (x, -oo, oo))
√2⋅√π
─────
2
  • examples on site are interactive so you can play with it without installing anything.

Michael #6: Starlette ASGI web framework

Extras:

Michael: PyCon 2019 dates out, put them on your calendar!

  • Tutorials: May 1-2 • Wednesday, Thursday
  • Talks and Events: May 3–5 • Friday, Saturday, Sunday
  • Sprints: May 6–9 • Monday through Thursday

Listener follow up on git pre-commit hooks util: pre-commit package

  • Matthew Layman, @mblayman
  • Heard the discussion about Git commit hooks at the end. I wanted to bring up pre-commit as an interesting project (written in Python!) that's useful for Git commit hooks.
  • tl;dr:
    • $ pip install pre-commit
    • $ ... create a .pre-commit-config.yaml
    • $ pre-commit install # This is a one time operation.
  • pre-commit's job is to manage a project's Git commit hooks. We use this on my team at work and the devs only need to run pre-commit install. This saves us from a bunch of failing CI builds where flake8 or other code style checks would fail.
  • We use pre-commit to run flake8 and black before allowing a commit to proceed. Some projects have a pre-commit configuration to use right out of the box (e.g., black https://github.com/ambv/black#version-control-integration).

Listener: You don't need that (pattern)

  • John Tocher
  • PyCon AU Talk Called "You don't need that” - by Christopher Neugebauer, it was an interesting take on why with a modern and powerful language like python, you may not need the conventionally described design patterns, ala the "Gang of four".

Sep 15 2018

23mins

Play

#114 What should be in the Python standard library?

Podcast cover
Read more

Sponsored by pythonbytes.fm/digitalocean

Brian #1: What should be in the Python standard library?

  • on lwn.net by Jake Edge
  • There was a discussion recently about what should be in the standard library, triggered by a request to add LZ4 compression.
  • Kinda hard to summarize but we’ll try:
    • Jonathan Underwood proposed adding LZ4 compression to stdlib.
    • Can of worms opened
    • zlib and bz2 already in stdlib
    • Brett proposed making something similar to hashlib for compression algorithms.
    • Against adding it:
      • lz4 not needed for stdlib, and actually, bz2 isn’t either, but it’s kinda late to remove.
    • PyPI is easy enough. put stuff there.
    • Led to a discussion of the role of stdlib.
      • If it’s batteries included, shouldn’t we add new batteries
      • Some people don’t have access to PyPI easily
      • Do we never remove elements? really?
      • Maybe we should have a lean stdlib and a thicker standard distribution of selected packages
        • who would decide?
        • same problem exists then of depending on it. How to remove stuff?
        • Steve Dower would rather see a smaller standard library with some kind of "standard distribution" of PyPI modules that is curated by the core developers.
      • A leaner stdlib could speed up Python version schedules and reduce burden on core devs to maintain seldom used packages.
    • See? can of worms.
    • In any case, all this would require a PEP, so we have to wait until we have a PEP process decided on.
Michael #2: Data Science portal for Home Assistant launched

  • via Paul Cutler
  • Home Assistant is launching a data science portal to teach you how you can learn from your own smart home data.
  • In 15 minutes you setup a local data science environment running reports.
  • A core principle of Home Assistant is that a user has complete ownership of their personal data. A users data lives locally, typically on the SD card in their Raspberry Pi
  • The Home Assistant Data Science website is your one-stop-shop for advice on getting started doing data science with your Home Assistant data.
  • To accompany the website, we have created a brand new Hass.io Add-on JupyterLab lite, which allows you to run a data science IDE called JupyterLab directly on your Raspberry Pi hosting Home Assistant. You do your data analysis locally, your data never leaves your local machine.
  • When you build something cool, you can share the notebook without the results, so people can run it at their homes too.
  • We have also created a Python library called the HASS-Data-Detective which makes it super easy to get started investigating your Home Assistant data using modern data science tools such as Pandas.
  • Check out the Getting Started notebook
  • IoT aside: I finally found my first IoT project: Recording in progress button.
Brian #3: What's the future of the pandas library?

  • Kevin Markham over at dataschool.io
  • pandas is gearing up to move towards a 1.0 release. Currently rc-ing 0.24
  • Plans are to get there “early 2019”.
  • Some highlights
    • method chaining - encouraged by core team
      • to encourage further, more methods will support chaining
    • Apache arrow likely to be part of pandas backend sometime after 1.0
    • Extension arrays - allow you to create custom data types
    • deprications
      • inplace parameter. It doesn’t work with chaining, doesn’t actually prevent copies, and causes codebase complexity
      • ix accessor, use loc and iloc instead
      • Panel data structure. Use MultiIndex instead
      • SparseDataFrame. Just use a normal DataFrame
      • legacy python support
Michael #4: PyOxidizer

  • PyOxidizer is a collection of Rust crates that facilitate building libraries and binaries containing Python interpreters.
  • PyOxidizer is capable of producing a single file executable - with all dependencies statically linked and all resources (like .pyc files) embedded in the executable
  • The Oxidizer part of the name comes from Rust: executables produced by PyOxidizer are compiled from Rust and Rust code is responsible for managing the embedded Python interpreter and all its operations.
  • PyOxidizer is similar in nature to PyInstaller, Shiv, and other tools in this space. What generally sets PyOxidizer apart is
    • Produced executables contain an embedded, statically-linked Python interpreter
    • have no additional run-time dependency on the target system
    • runs everything from memory (as opposed to e.g. extracting Python modules to a temporary directory and loading them from there).
Brian #5: Working With Files in Python

  • by Vuyisile Ndlovu on RealPython
  • Very comprehensive write up on working with files and directories
  • Includes legacy and modern methods.
    • Pay attention to pathlib parts if you are using 3.4 plus
    • Also great for “if you used to do x, here’s how to do it with pathlib”.
  • Included:
    • Directory listings
    • getting file attributes
    • creating directories
    • file name pattern matching
    • traversing directories doing stuff with the files in there
    • creating temp directories and files
    • deleting, copying, moving, renaming
    • archiving with zip and tar including reading those
    • looping over files
Michael #6: $ python == $ python3?

  • via David Furphy
  • Homebrew tried this recently & got "persuaded" to reverse.
  • Also in recent discussion of edits to PEP394, GvR said absolutely not now, probably not ever.
  • Guido van Rossum
    • RE: python doesn’t exist on macOS as a command: Did you mean python2 there? In my experience macOS comes with python installed (and invoking Python 2) but no python2 link (hard or soft). In any case I'm not sure how this strengthens your argument.
    • I'm also still unhappy with any kind of endorsement of python pointing to python3. When a user gets bitten by this they should receive an apology from whoever changed that link, not a haughty "the PEP endorses this".
    • Regardless of what macOS does I think I would be happier in a future where python doesn't exist and one always has to specify python2 or python3. Quite possibly there will be an age where Python 2, 3 and 4 all overlap, and EIBTI.
Extras:

Michael: A letter to the Python community in Africa

  • via Anthony Shaw
  • Believe the broader international Python and Software community can learn a lot from what so many amazing people are doing across Africa.
  • e.g. The attendance of PyCon NA was 50% male and 50% female.
Joke:
via Luke Russell:
A: “Knock Knock”
B: “Who’s There"
A: ……………………………………………………………………………………….“Java”

Also: Java 4EVER video is amazing: youtube.com/watch?v=kLO1djacsfg

Jan 26 2019

28mins

Play

#68 Python notebooks galore!

Podcast cover
Read more

Sponsored by DigitalOcean! http://do.co/python

Brian #1: dumb-pypi

  • This takes some fiddling with and trial and error. I definitely need to write up my experiences with this as a blog post.
  • Combine with pip download (covered in episode 24), this makes it super easy to create a static locally hosted pypi server, either for all of your packages, or for your proprietary packages.
  • Roughly:
pip download -d my-packages-dir [HTML_REMOVED]
ls my-packages-dir > package-list.txt
dumb-pypi --package-list my-packages-dir \
--packages-url [HTML_REMOVED] \
--output-dir my-pypi
  • Now add something like this to requirements.txt or pip commands:
  • --trusted-host [HTML_REMOVED] -i http://[HTML_REMOVED]/my-pypi/simple

Michael #2: Requests-HTML: HTML Parsing for Humans

  • This library intends to make parsing HTML (e.g. scraping the web) as simple and intuitive as possible.
  • When using this library you automatically get:
    • Full JavaScript support!
    • CSS Selectors (a.k.a jQuery-style, thanks to PyQuery).
    • XPath Selectors, for the faint at heart.
    • Mocked user-agent (like a real web browser).
    • Automatic following of redirects.
    • Connection–pooling and cookie persistence.
    • The Requests experience you know and love, with magical parsing abilities

Brian #3: A phone number proxy

  • Naomi Pentrel, @naomi_pen on twilio blog
  • Set up a phone number that you can share for temporary events to send and receive texts that get forwarded to your actual number.

Michael #4: Notebooks galore part 1: Datalore

  • In cloud and ready to go
  • Intelligent code editor
  • Out-of-the-box Python tools
  • Collaboration
  • Integrated version control
  • Incremental calculations: Improve and adjust models without hustling with additional recalculations. Datalore follows dependencies between multiple computations and automatically applies relevant recalculations.

Brian #5: bellybutton

  • by Chase Stevens, @hchasestevens
  • Tool for creating personal static analysis/style tools like pycodestyle, pylint, and flake8
  • Teams often have some of their own style requirements that can’t be expressed as flake8 flags and exceptions.
  • Example: deprecating internal library functions and catching that by the linter.

Michael #6:Notebooks galore part 2

  • Python 3.6 Jupyter Notebook on Azure
  • Google Colaboratory
  • JupyterLab is Ready for Users
    • JupyterLab is an interactive development environment for working with notebooks, code and data. Most importantly, JupyterLab has full support for Jupyter notebooks. Additionally, JupyterLab enables you to use text editors, terminals, data file viewers, and other custom components side by side with notebooks in a tabbed work area.
  • you can pip install python packages within python code itself.
    • Super useful in situation #1 when you need a package that's not included but you don't have access to the shell.
    • If you need to upgrade a package. For example the Pandas version is a little old on Azure, so you can upgrade by simply running:
import pip
pip.main(['install', 'pandas', '--upgrade'])

Mar 06 2018

19mins

Play

#107 Restructuring and searching data, the Python way

Podcast cover
Read more

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean

Brian #1: glom: restructuring data, the Python way

  • glom is a new approach to working with data in Python, featuring:
    • Path-based access for nested structure
      • data\['a'\]['b']['c']glom(data, 'a.b.c')
    • Declarative data transformation using lightweight, Pythonic specifications
      • glom(target, spec, **kwargs) with options such as
        • a default value if value not found
        • allowed exceptions
    • Readable, meaningful error messages:
      • PathAccessError: could not access 'c', part 2 of Path('a', 'b', 'c') is better than
      • TypeError: 'NoneType' object is not subscriptable
    • Built-in data exploration and debugging features

Michael #2: Scientific GUI apps with TraitsUI

Brian #3: Pampy: The Pattern Matching for Python you always dreamed of

  • “Pampy is pretty small (150 lines), reasonably fast, and often makes your code more readable and hence easier to reason about.”
  • uses _ as the missing info in a pattern
  • simple match signature of match(input, pattern, action)

  • Examples

    • nested lists and tuples
from pampy import match, _

x = [1, [2, 3], 4]
match(x, [1, [_, 3], _], lambda a, b: [1, [a, 3], b]) # => [1, [2, 3], 4]
- dicts:
pet = { 'type': 'dog', 'details': { 'age': 3 } }
match(pet, { 'details': { 'age': _ } }, lambda age: age) # => 3
match(pet, { _ : { 'age': _ } }, lambda a, b: (a, b)) # => ('details', 3)

Michael #4: Google AI better than doctors at detecting breast cancer

  • Google’s deep learning AI called LYNA able to correctly identify tumorous regions in lymph nodes 99 per cent of the time.
  • We think of the impact of AI as killing 'low end' jobs [see poster], but these are "doctor" level positions.
  • The presence or absence of these ‘nodal metastases’ influence a patient’s prognosis and treatment plan, so accurate and fast detection is important.
  • In a second trial, six pathologists completed a diagnostic test with and without LYNA’s assistance. With LYNA’s help, the doctors found it ‘easier’ to detect small metastases, and on average the task took half as long.

Brian #5: 2018 Advent of Code

Another winter break activity people might enjoy is practicing with code challenges. AoC is a fun tradition.

  • a calendar of small programming puzzles for a variety of skill sets and skill levels that can be solved in any programming language you like.
  • don't need a computer science background to participate
  • don’t need a fancy computer; every problem has a solution that completes in at most 15 seconds on ten-year-old hardware.
  • There’s a leaderboard, so you can compete if you want. Or just have fun.
  • Past years available, back to 2015.
  • Some extra tools and info: awesome-advent-of-code

Michael #6: Red Hat Linux 8.0 Beta released, now (finally) updated to use Python 3.6 as default instead of 2.7

  • First of all, my favorite comment was a correction to the title: legacy python *
  • Python 3.6 is the default Python implementation in RHEL 8; limited support for Python 2.7 is provided. No version of Python is installed by default.“
    • Red Hat Enterprise Linux 8 is distributed with Python 3.6. The package is not installed by default. To install Python 3.6, use the yum install python3 command.
    • Python 2.7 is available in the python2 package. However, Python 2 will have a shorter life cycle and its aim is to facilitate smoother transition to Python 3 for customers.
    • Neither the default python package nor the unversioned /usr/bin/python executable is distributed with RHEL 8. Customers are advised to use python3 or python2 directly. Alternatively, administrators can configure the unversioned python command using the alternatives command.
  • Python scripts must specify major version in hashbangs at RPM build time
    • In RHEL 8, executable Python scripts are expected to use hashbangs (shebangs) specifying explicitly at least the major Python version.

Extras:

Michael: We were featured on TechMeme Long Ride Home podcast. Check out their podcast here. Thank you to Brian McCullough, the host of the show. I just learned about their show through this exchange but can easily see myself listening from time to time. It’s like Python Bytes, but for the wider tech world and less developer focused but still solid tech foundations.

Brian: First story was about glom. I had heard of glom before, but got excited after interviewing Mahmoud for T&C 55, where we discussed the difficulty in testing if you use glom or DSLs in general. A twitter exchange and GH issue followed the episode, with Anthony Shaw. At one point, Ant shared this great joke from Brenan Kellar:

A QA engineer walks into a bar. Orders a beer. Orders 0 beers. Orders 99999999999 beers. Orders a lizard. Orders -1 beers. Orders a ueicbksjdhd.

First real customer walks in and asks where the bathroom is. The bar bursts into flames, killing everyone.

— Brenan Keller (@brenankeller) November 30, 2018

Dec 07 2018

22mins

Play

#113 Python Lands on the Windows 10 App Store

Podcast cover
Read more

Sponsored by https://pythonbytes.fm/digitalocean

Brian #1: Advent of Code 2018 Solutions

  • Michael Fogleman
  • Even if you didn’t have time or energy to do the 2018 AoC, you can learn from other peoples solutions. Here’s one set written up in a nice blog post.
Michael #2: Python Lands on the Windows 10 App Store

  • Python Software Foundation recently released Python 3.7 as an app on the official Windows 10 app store.
  • Python 3.7 is now available to install from the Microsoft Store, meaning you no longer need to manually download and install the app from the official Python website.
  • there is one limitation. “Because of restrictions on Microsoft Store apps, Python scripts may not have full write access to shared locations such as TEMP and the registry.
  • Discussed with Steve Dower over on Talk Python 191
Brian #3: How I Built A Python Web Framework And Became An Open Source Maintainer

  • Florimond Manca
  • Bocadillo - “A modern Python web framework filled with asynchronous salsa”
  • maintaining an open source project is a marathon, not a sprint.”
  • Tips at the end of the article include tips for the following topics, including recommendations and tool choices:
    • Project definition
    • Marketing & Communication
    • Community
    • Project management
    • Code quality
    • Documentation
    • Versioning and releasing
Michael #4: Python maintainability score via Wily

  • via Anthony Shaw
  • A Python application for tracking, reporting on timing and complexity in tests
  • Easiest way to calculate it is with wily https://github.com/tonybaloney/wily … the metrics are ‘maintainability.mi’ and ‘maintainability.rank’ for a numeric and the A-F scale.
    • Build an index: wily build src
    • Inspect report: wily report file
    • Graph: wily graph file metric
Brian #5: A couple fun awesome lists

  • Awesome Python Security resources
    • Tools
      • web framework hardening, ex: secure.py
      • multi tools
      • static code analysis, ex: bandit
      • vulnerabilities and security advisories
      • cryptography
      • app templates
    • Education
      • lots of resources for learning
    • Companies
  • Awesome Flake8 Extensions
    • clean code
    • testing, including
    • security
    • documentation
    • enhancements
    • copyrights
Michael #6: fastlogging

  • via Robert Young
  • A faster replacement of the standard logging module with a mostly compatible API.
  • For a single log file it is ~5x faster and for rotating log file ~13x faster.
  • It comes with the following features:
    • (colored, if colorama is installed) logging to console
    • logging to file (maximum file size with rotating/history feature can be configured)
    • old log files can be compressed (the compression algorithm can be configured)
    • count same successive messages within a 30s time frame and log only once the message with the counted value.
    • log domains
    • log to different files
    • writing to log files is done in (per file) background threads, if configured
    • configure callback function for custom detection of same successive log messages
    • configure callback function for custom message formatter
    • configure callback function for custom log writer

Joke: >>> import antigravity

Jan 18 2019

23mins

Play

#102 Structure of a Flask Project

Podcast cover
Read more

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean

Brian #1: QuantEcon

  • “Open source code for economic modeling”
  • “QuantEcon is a NumFOCUS fiscally sponsored project dedicated to development and documentation of modern open source computational tools for economics, econometrics, and decision making.”
  • Educational resource that includes:
    • Lectures, workshops, and seminars
    • Cheatsheets for scientific programming in Python and Julia
    • Notebooks
    • QuantEcon.py : open source Python code library for economics

Michael #2: Structure of a Flask Project

  • Flask is very flexible, it has no certain pattern of a project folder structure. Here are some suggestions.
  • I always keep this one certain rule when writing modules and packages:
  • Candidate structure:
project/
__init__.py
models/
__init__.py users.py posts.py
...
routes/
__init__.py home.py account.py dashboard.py
...
templates/
base.html
post.html
...
services/
__init__.py google.py mail.py
  • Love it! To this, I would rename routes to views or controllers and add a viewmodels folder and viewmodels themselves.
  • Brian, see anything missing?
    • ya. tests. :)
  • Another famous folder structure is app based structure, which means things are grouped bp application
  • I (Michael) STRONGLY recommend Flask blueprints

Brian #3: Overusing lambda expressions in Python

  • lambda expressions vs defined functions
    1. They can be immediately passed around (no variable needed)
    2. They can only have a single line of code within them
    3. They return automatically
    4. They can’t have a docstring and they don’t have a name
    5. They use a different and unfamiliar syntax
  • misuses:
    • naming them. Just write a function instead
    • calling a single function with a single argument : just use that func instead
  • overuse:
    • if they get complex, even a little bit, they are hard to read
    • has to be all on one line, which reduces readibility
    • map and filter : use comprehensions instead
    • using custom lambdas instead of using operators from the operator module.

Michael #4: Asyncio in Python 3.7

  • by Cris Medina
  • The release of Python 3.7 introduced a number of changes into the async world.
    • Some may even affect you even if you don’t use asyncio.
  • New Reserved Keywords: The async and await keywords are now reserved.
    • There’s already quite a few modules broken because of this. However, the fix is easy: rename any variables and parameters.
  • Context Variables: Version 3.7 now allows the use of context variables within async tasks. If this is a new concept to you, it might be easier to picture it as global variables whose values are local to the currently running coroutines.
  • Python has similar constructs for doing this very thing across threads. However, those were not sufficient in async-world
  • New asyncio.run() function
    • With a call to asyncio.run(), we can now automatically create a loop, run a task on it, and close it when complete.
  • Simpler Task Management: Along the same lines, there’s a new asyncio.create_task() function that helps make tasks that inside the current loop, instead of having to get the loop first and calling create task on top of it.
  • Simpler Event Loop Management: The addition of asyncio.get_running_loop() will help determine the active event loop, and catch a RuntimeError if there’s no loop running.
  • Async Context Managers: Another quality-of-life improvement. We now have the asynccontextmanager() decorator for producing async context managers without the need for a class that implements __aenter__() or __aexit__().
  • Performance Improvements: Several functions are now optimized for speed, some were even reimplemented in C. Here’s the list:
    • asyncio.get_event_loop() is now 15 times faster.
    • asyncio.gather() is 15% faster.
    • asyncio.sleep() is two times faster when the delay is zero or negative.
    • asyncio.Future callback management is optimized.
    • Reduced overhead for asyncio debug mode.
  • Lots lots more

Brian #5: Giving thanks with **pip thank**

Michael #6: Getting Started With Testing in Python

  • by Anthony Shaw, 33 minutes reading time according to Instapaper
  • Automated vs. Manual Testing
  • Unit Tests vs. Integration Tests: A unit test is a smaller test, one that checks that a single component operates in the right way. A unit test helps you to isolate what is broken in your application and fix it faster.
  • Compares unittest, nose or nose2, pytest
  • Covers things like:
    • Writing Your First Test
    • Where to Write the Test
    • How to Structure a Simple Test
    • How to Write Assertions
    • Dangers of Side Effects
  • Testing in PyCharm and VS Code
  • Testing for Web Frameworks Like Django and Flask
  • Advanced Testing Scenarios
  • Even: Testing for Security Flaws in Your Application

Extras:

Oct 31 2018

26mins

Play

#76 Goodbye zero-versioning

Podcast cover
Read more

Sponsored by Datadog: pythonbytes.fm/datadog

Brian #1: Unlearning toxic behaviors in a code review culture

  • unhelpful behaviors:
    • passing off opinion as fact
    • overwhelming with an avalanche of comments
    • asking people to fix problems they didn’t cause “while they’re at it”.
    • asking judgmental questions
    • being sarcastic
    • using emojis
    • not replying to comments
    • ignoring (not calling out) toxic behavior from high performers
  • helpful:
    • use questions or recommendations to drive dialog
    • collaborate, don’t back-seat drive
    • respond to every comment
    • know when to take a discussion offline
    • use opportunities to teach, and don’t show off
    • don’t show surprise of lack of knowledge by others
    • automate what can be
    • refuse to normalize toxic behavior
    • managers: hire carefully, listen to your team, and enforce
    • set the standard as your team is small and growing
    • understand you might be part of the problem

Michael #2: Flask 1.0 Released

  • Dropped support for Python 2.6 and 3.3.
  • The CLI is more flexible. FLASK_APP can point to an app factory, optionally with arguments. It understands import names in more cases where filenames were previously used. It automatically detects common filenames, app names, and factory names. FLASK_ENV describes the environment the app is running in, like development, and replaces FLASK_DEBUG in most cases. See the docs to learn more.
  • If python-dotenv is installed, the flask CLI will load environment variables from .flaskenv and .env files rather than having to export them in each new terminal.
  • The development server is multi-threaded by default to handle concurrent requests during development.
  • flask.ext, which was previously deprecated, is completely removed. Import extensions by their actual package names.
  • Accessing missing keys from request.form shows a more helpful error message in debug mode, addressing a very common source of confusion for developers.
  • Error handlers are looked up by code then exception class, on the blueprint then application. This gives more predictable control over handlers, including being able to handle HTTPException.
  • The behavior of app.logger has been greatly simplified and should be much easier to customize. The logger is always named flask.app, it only adds a handler if none are registered, and it never removes existing handlers. See the docs to learn more.
  • The test_client gained a json argument for posting JSON data, and the Response object gained a get_json method to decode the data as JSON in tests.
  • A new test_cli_runner is added for testing an app's CLI commands.
  • Many documentation sections have been rewritten to improve clarity and relevance. This is an ongoing effort.
  • The tutorial and corresponding example have been rewritten. They use a structured layout and go into more detail about each aspect in order to help new users avoid common issues and become comfortable with Flask.
    • There are many more changes throughout the framework. Read the full

Brian #3: So, I still don’t quite get pipenv, ….

  • Best discussion of why pipenv is useful for applications I’ve come across so far is Pipenv: A Guide to the New Python Packaging Tool
  • Starts with a discussion of situations where pip, pip freeze, and requirements.txt fall apart.
    • requirements.txt often just have an applications direct dependencies, not sub-dependencies.
    • pip freeze > requirements.txt will pin your versions to specific versions, but then you’ve got to keep track of dependencies and sub-dependencies.
    • Pipfile intends to replace requirements.txt, with a simple-ish human readable format. Also includes extra things like dev environment support.
    • Pipfile.lock intends to replace pinned requirements.txt files. Also includes hashes to validate versions haven’t been corrupted.
    • pipenv also includes cool tools like dependency graphing, checking for updates, etc.
  • pipenv should be used for applications, but not packages intended to be included in other applications. But you can use it during package development, just probably not include the Pipfile and Pipfile.lock in the repo or package distribution. - Brian’s comment

Bonus extra:

Michael #4: GraalVM: Run Programs Faster Anywhere

  • Why?
    • Current production virtual machines (VMs) provide high performance execution of programs only for a specific language or a very small set of languages.
    • Compilation, memory management, and tooling are maintained separately for different languages, violating the ‘don’t repeat yourself’ (DRY) principle.
    • high performance VMs are heavyweight processes with high memory footprint and difficult to embed.
  • Oracle Labs started a new research project for exploring a novel architecture for virtual machines. Our vision was to create a single VM that would provide high performance for all programming languages, therefore facilitating communication between programs.
  • Released: GraalVM, a universal virtual machine designed for a polyglot world.
  • GraalVM provides high performance for individual languages and interoperability with zero performance overhead for creating polyglot applications.
  • GraalVM 1.0 allows you to run:
    • JVM-based languages like Java, Scala, Groovy, or Kotlin
    • JavaScript (including Node.js)
    • LLVM bitcode (created from programs written in e.g. C, C++, or Rust)
    • Experimental versions of Ruby, R, and Python
  • Future: This first release is only the beginning. We are working on improving all aspects of GraalVM; in particular the support for Python

Brian #5: Testing a Flask Application using pytest

  • Small demo project, and article, that teaches the use of pytest in Flask.
  • unit testing and functional testing
  • Article covers testing models, with an example of a new user.
  • project also has examples of using a test client to check the login page, password authentication, and a lot more.
  • Very cool project.

Bonus: A cool new pytest plugin: pytest-caprng

  • Tests that use random or np.random may fail, but when you re-run them, they don’t fail, which makes them hard to debug.
  • This plugin adds pytest flags --caprng-global-stdlib and --caprng-global-np, which saves the random state before each test so that if you re-run the test, the random-ness is not so random, and you can reproduce your failure.
  • Also, thanks John for reminding me what “stochastic” means.

Michael #6: How to have a great first PyCon

  • Spending your time: which talks should I go to? The talks at PyCon are typically uploaded to YouTube within 24 hours after the talk ends. I am suggesting that you don’t need to worry about attending every talk.
  • Open spaces: attend them and consider hosting your own!
  • There are a few reasons I often pick open spaces over talks:
    • Often the open spaces are more niche and topical than the talks: there are some subjects that exist in open spaces every year but which I’ve never seen a talk on
    • Open spaces are all about interaction and discussion whereas talks are a monologue that often evolves into subsequent dialogues
    • Open spaces aren’t recorded whereas the talks are, meaning you can’t really catch up on them later
  • Tips for starting conversation, breakfast and lunch time…
  • The hallway track 👣
    • Something you might consider doing while at PyCon is taking breaks in the hallway.
    • In addition to joining or starting a table in the hallway, consider identifying groups that have a PacMan opening to join and make sure the groups you’re in are PacMan-friendly.
  • Interacting online during PyCon 🐦
    • I recommend getting a Twitter account to make it easier to passively keep up with folks from PyCon after the conference ends.
    • Sometimes people on Twitter will ask if anyone would like to join them for dinner and you might decide to reply and say you’d like to join.
  • Networking isn’t a dirty word: it means making friends 👥
  • I hear two opposing concerns sometimes expressed about PyCon:
    • Isn’t everyone here to get a job or hire people?
    • Is it acceptable to go to PyCon looking for a job?
    • PyCon is a networking event. That doesn’t necessarily mean everyone is there to get a job, but it also definitely doesn’t mean it’s unacceptable to job-seek at Python.
  • Other topics include
    • Volunteering
    • Evening events: dinners and board games
    • Give a lightning talk ⚡
    • Take care of yourself
  • Final tip from commentor: If you are on windows, it's helpful to install a virtual image of a linux like the current ubuntu on your laptop, because you could run into situations where you want to follow a talk / training which doesn't work on windows and then you're missing a great opportunity to learn.

Our news

  • Come see us at PyCon!!! We’ll have stickers!
  • Brian’s talk is Friday at 5 something.
  • We are doing a live Python Bytes open session, join “friends of the show” to get notified
  • I’ll be at Microsoft BUILD too
  • PyGotham 2018 Call for Proposals
  • http://PyCon.DE (24-26 October 2018 in Karlsruhe, Germany) starting our CfP tomorrow until May 20. http://de.pycon.org

May 04 2018

30mins

Play

#98 Python-Electron as a Python GUI

Podcast cover
Read more

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean

Brian #1: Making Etch-a-Sketch Art With Python

  • Really nice write up of methodically solving problems with simplifying the problem space, figuring out what parts need solved, grabbing off the shelf bits that can help, and putting it all together.
  • Plus it would be a fun weekend (or several) project with kids helping.
  • Controlling the Etch-a-Sketch
    • Raspberry Pi, motors, cables, wood fixture
    • Software to control the motors
  • Picture simplification with edge detection with Canny edge detection.
  • Lines to motor control with path finding with networkx library.
  • Example results included in article.
  • Pentium song: https://www.youtube.com/watch?v=qpMvS1Q1sos

Michael #2: Dropbox moves to Python 3

  • They just rolled out one of the largest Python 3 migrations ever
  • Dropbox is one of the most popular desktop applications in the world
  • Much of the application is written using Python. In fact, Drew’s very first lines of code for Dropbox were written in Python for Windows using venerable libraries such as pywin32.
  • Though we’ve relied on Python 2 for many years (most recently, we used Python 2.7), we began moving to Python 3 back in 2015.
  • If you’re using Dropbox today, the application is powered by a Dropbox-customized variant of Python 3.5.
  • Why Python 3?
    • Exciting new features: Type annotations and async & await
    • Aging toolchains: As Python 2 has aged, the set of toolchains initially compatible for deploying it has largely become obsolete
  • Embedding Python
    • To solve build and deploy problem, we decided on a new architecture to embed the Python runtime in our native application.
    • Deep integration with the OS (e.g. smart sync) means native apps are required
  • In future posts, we’ll look at:
    • How we report crashes on Windows and macOS and use them to debug both native and Python code.
    • How we maintained a hybrid Python 2 and 3 syntax, and what tools helped.
    • Our very best bugs and stories from the Python 3 migration.

Brian #3: Resources for PyCon that relate to really any talk venue

Michael #4: Electron as GUI of Python Applications

  • via Andy Bulka
  • Electron Python is a template of code where you use Electron (nodejs + chromium) as a GUI talking to Python 3 as a backend via zerorpc. Similar to Eel but much more capable e.g. you get proper native operating system menus — and users don’t need to have Chrome already installed.
  • Needs to run zerorpc server and then start electron separately — can be done via the node backend
  • using Electron as a GUI toolkit gets you
    • native menus, notifications
    • installers, automatic updates to your app
    • debugging and profiling that you are used to, using the Chrome debugger
    • ES6 syntax (a cleaner Javascript with classes, module imports, no need for semicolons etc.). Squint, look sideways, and it kinda looks like Python… ;-)
    • the full power of nodejs and its huge npm package repository
    • the large community and ecosystem of Electron
  • How to package this all?
  • Building a deployable Python-Electron App post by Andy Bulka
    • One of the great things about using Electron as a GUI for Python is that you get to use cutting edge web technologies and you don’t have to learn some old, barely maintained GUI toolkit
    • How much momentum, money, time and how many developer minds are focused on advancing web technologies? Answer: it’s staggeringly huge.
    • Compare this with the number of people maintaining old toolkits from the 90’s e.g. wxPython? Answer: perhaps one or two people in their spare time.
    • Which would you rather use?
    • Final quote: And someone please wrap Electron-Python into an IDE so that in the future all we have to do is click a ‘build’ button — like we could 20 years ago. :-)

Brian #5: pluggy: A minimalist production ready plugin system

  • docs
  • plugin management and hook system used by pytest
  • A separate package to allow other projects to include plugin capabilities without exposing unnecessary state or behavior of the host project.

Michael #6: How China Used a Tiny Chip to Infiltrate U.S. Companies

  • via Eduardo Orochena
  • The attack by Chinese spies reached almost 30 U.S. companies, including Amazon and Apple, by compromising America’s technology supply chain, according to extensive interviews with government and corporate sources.
  • In 2015, Amazon.com Inc. began quietly evaluating a startup called Elemental Technologies, a potential acquisition to help with a major expansion of its streaming video service, known today as Amazon Prime Video. (from Portland!)
  • To help with due diligence, AWS, which was overseeing the prospective acquisition, hired a third-party company to scrutinize Elemental’s security
  • servers were assembled for Elemental by Super Micro Computer Inc., a San Jose-based company (commonly known as Supermicro) that’s also one of the world’s biggest suppliers of server motherboards
  • Nested on the servers’ motherboards, the testers found a tiny microchip, not much bigger than a grain of rice, that wasn’t part of the boards’ original design.
  • Amazon reported the discovery to U.S. authorities, sending a shudder through the intelligence community. Elemental’s servers could be found in Department of Defense data centers, the CIA’s drone operations, and the onboard networks of Navy warships. And Elemental was just one of hundreds of Supermicro customers.
  • During the ensuing top-secret probe, which remains open more than three years later, investigators determined that the chips allowed the attackers to create a stealth doorway into any network that included the altered machines. Multiple people familiar with the matter say investigators found that the chips had been inserted at factories run by manufacturing subcontractors in China.
  • One government official says China’s goal was long-term access to high-value corporate secrets and sensitive government networks. No consumer data is known to have been stolen.
  • American investigators eventually figured out who else had been hit. Since the implanted chips were designed to ping anonymous computers on the internet for further instructions, operatives could hack those computers to identify others who’d been affected.

Extra:

Oct 08 2018

26mins

Play

#153 Auto format my Python please!

Podcast cover
Read more

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean

Michael #1: Building a Python C Extension Module

  • Tutorial, learn to use the Python API to write Python C extension modules.
  • And
    • Invoke C functions from within Python
    • Pass arguments from Python to C and parse them accordingly
    • Raise exceptions from C code and create custom Python exceptions in C
    • Define global constants in C and make them accessible in Python
    • Test, package, and distribute your Python C extension module
  • Extending Your Python Program
    • there may be other lesser-used system calls that are only accessible through C
  • Steps: Writing a Python Interface in C
    • Figure out the arguments (e.g. int fputs(const char *, FILE *) )
    • Implement in C:
#include Python.h
static PyObject *method_fputs(PyObject *self, PyObject *args) {
char *str, *filename = NULL;
int bytes_copied = -1;
/* Parse arguments */
if(!PyArg_ParseTuple(args, "ss", &str, &filename)) {
return NULL;
}
FILE *fp = fopen(filename, "w");
bytes_copied = fputs(str, fp);
fclose(fp);
return PyLong_FromLong(bytes_copied);
}
  • In line 2, you declare the argument types you wish to receive from your Python code
  • line 6, then you’ll see that PyArg_ParseTuple() copies into the char*’s
  • Write regular C code (fopen, fputs)
  • Return: PyLong_FromLong() creates a PyLongObject, which represents an integer object in Python.
  • a few extra functions that are necessary
    • write definitions of your module and the methods it contains
    • Before you can import your new module, you first need to build it. You can do this by using the Python package distutils.

Brian #2: What’s New in Python 3.8 - docs.python.org

We’ve already talked about the big hitters:

  • assignment expressions, (the walrus operator)
  • positional only parameters, (the / in the param list)
  • f-strings support = for self-documenting expressions and debugging

There are a few more goodies I wanted to quickly mention:

  • More async: python -m asyncio launches a native async REPL
  • More helpful warnings and messages when
    • using is and is not to compare strings and integers and other types intended to be compared with == and !=
    • Missing the comma in stuff like [(1,2) (3,4)]. Happens all the time with parametrized testing
  • you can do iterable unpacking in a yield or return statement
    • x = (1, 2, 3)
    • a, *b = x
    • return a, *b <- this used to be a syntax error
      • you had to do return (a, *b)
  • New module importlib.metadata lets you access things like version numbers or dependent library required version numbers, and cool stuff like that.

Michael #3: UK National Cyber Security Centre (NCSC) is warning developers of the risks of sticking with Python 2.7, particularly for library writers

  • NCSC likens companies continuing to use Python 2 past its EOL to tempting another WannaCry or Equifax incident.
    • Equifax details: a vulnerability, dubbed CVE-2017-5638, was discovered in Apache Struts, an open source development framework for creating enterprise Java applications that Equifax, along with thousands of other websites, uses…
  • Quote: "If you're still using 2.x, it's time to port your code to Python 3," the NCSC said. "If you continue to use unsupported modules, you are risking the security of your organisation and data, as vulnerabilities will sooner or later appear which nobody is fixing."
  • Moreover: "If you maintain a library that other developers depend on, you may be preventing them from updating to 3," the agency added. "By holding other developers back, you are indirectly and likely unintentionally increasing the security risks of others.”
  • "If migrating your code base to Python 3 is not possible, another option is to pay a commercial company to support Python 2 for you," the NCSC said.
  • NCSC: If you don't migrate, you should expect security incidents
  • Python's popularity makes updating code imperative: The reason the NCSC is warning companies about Python 2's impending EOL is because of the language's success.

Brian #4: Pythonic News

  • Sebastian A. Steins
  • “A Hacker News lookalike written in Python/Django”
  • “ powering https://news.python.sc"
  • Cool that it’s open source, and on github
  • Was submitted to us by Sebastian, and a few others too, so there is excitement.
  • It’s like 6 days old and has 153 stars on github, 4 contributors, 18 forks.
  • Fun.

Michael #5: Deep Learning Workstations, Servers, Laptops, and GPU Cloud

  • GPU-accelerated with TensorFlow, PyTorch, Keras, and more pre-installed. Just plug in and start training. Save up to 90% by moving off your current cloud and choosing Lambda.
  • They offer:
    • TensorBook: GPU Laptop for $2,934
    • Lambda Quad: 4x GPU Workstation for $21,108 (yikes!)
    • All in: Lambda Hyperplane: 8x Tesla V100 Server, starting at $114,274
  • But compare to:
    • AWS EC2: p3.8xlarge @ $12.24 per Hour => $8,935 / month

Brian #6: Auto formatters for Python

  • A comparison of autopep8, yapf, and black
  • Auto formatters are super helpful for teams. They shut down the unproductive arguments over style and make code reviews way more pleasant. People can focus on content, not being the style police.
  • We love black. But it might be a bit over the top for some people. Here are a couple of other alternatives.
  • autopep8 - mostly focuses on PEP8
    • “autopep8 automatically formats Python code to conform to the PEP 8 style guide. It uses the pycodestyle utility to determine what parts of the code needs to be formatted. autopep8 is capable of fixing most of the formatting issues that can be reported by pycodestyle.”
  • black - does more
    • doesn’t have many options, but you can alter line length, can turn of string quote normalization, and you can limit or focus the files it sees.
    • does a cool “check that the reformatted code still produces a valid AST that is equivalent to the original.” but you can turn that off with --fast
  • yapf - way more customizable.
    • Great if you want to auto format to a custom style guide.
    • “The ultimate goal is that the code YAPF produces is as good as the code that a programmer would write if they were following the style guide. It takes away some of the drudgery of maintaining your code.”
  • Article is cool in that it shows some sample code and how it’s changed by the different formatters.

Extras:

Michael:

  • New courses coming
  • Financial Aid Launches for PyCon US 2020!

Joke:

  • American Py Song

  • From Eric Nelson:

    • Math joke. “i is as complex as it gets. jk.”

Oct 23 2019

26mins

Play

#57 Our take on Excel and Python

Podcast cover
Read more

Sponsored by DigitalOcean: http://digitalocean.com

Brian #1: Testing Python 3 and 2 simultaneously with retox

  • Anthony Shaw
  • tox allows you to run the same tests in multiple configurations.
    • For example, multiple Python interpreters (2 vs 3), or on different hardware, or using different options, etc.
    • tox can also tests your packaging code (on by default, but can be disabled)
  • detox allows multiple configurations to be tested in parallel with multiprocessing
    • typically running all tests 2-4 times faster
  • retox does this with a GUI
    • also adds “watch” capability

Michael #2: Robo 3T / RoboMongo

  • MongoDB GUI with embedded shell
  • CLI interaction
  • GUI when you want it
  • No. 34 repository on GitHub

Brian #3: regular expressions

  • Regular Expressions Practical Guide
    • Python examples for some common expressions
    • How to use the built in re package for email addresses, URLs, phone numbers
    • substitution with re.sub()
    • splitting a string with re.split()
    • what some of the escape shortcuts mean, like \w for word, \s for whitespace, etc.
    • iterating through matches with re.finditer()
    • Using compiled expressions
  • Regular Expressions for Data Scientists
    • another great intro, that also talks about:
    • re.search()
    • re.findall()
    • match groups

Michael #4: MongoEngine

  • MongoEngine is a Document-Object Mapper (think ORM, but for document databases) for working with MongoDB from Python.
  • Map classes to MongoDB (think SQLAlchemy but for document databases)
  • Adds features lacking from MongoDB
    • Schema
    • Required fields
    • Constraints
    • Relationships

Brian #5: Introducing PrettyPrinter for Python

  • a powerful, syntax-highlighting, and declarative pretty printer for Python 3.6
  • goals
    • Implement an algorithm that tries very hard to produce pretty output, even if it takes a bit more work.
    • Implement a dead simple, declarative interface to writing your own pretty printers. Python developers rarely write __repr__ methods because they're a pain; no one will definitely write pretty printing rules for user-defined types unless it's super simple.
    • Implement syntax-highlighting that doesn't break on invalid Python syntax.

Michael #6: Excel and Python

  • Replace VBA
  • Python in Excel as the main scripting language
  • They need feedback (fill out their survey, upvote the issue)

Our news

Michael:

Dec 21 2017

15mins

Play

#80 Dan Bader drops by and we found 30 new Python projects

Podcast cover
Read more

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean

Brian #1: Packaging Python Projects

Dan #2: gidgethub — An async library for calling GitHub’s API

  • Talk to GitHub API to add/modify issues, pull-requests, comments, …
  • Also helpers to parse GitHub’s webhook events so you can write bots that react to new issues, comments, commits etc.
  • Used it in @Mariatta’s GitHub Bot tutorial:https://github.com/Mariatta/github-bot-tutorial
  • Cool architecture for a “modern” Python web API library (async, sansio, decorator based event callbacks)
    • supports different async backends: aiohttp, treq, Tornado
      • sans-I/O: “protocol implementations written in Python that perform no I/O (this means libraries that operate directly on text or bytes)”
      • Why? → “reusability. By implementing network protocols without any I/O and instead operating on bytes or text alone, libraries allow for reuse by other code regardless of their I/O decisions. In other words by leaving I/O out of the picture a network protocol library allows itself to be used by both synchronous and asynchronous I/O code”
  • (Biggest issue in that workshop was getting everyone upgraded to Python 3.6…but more on that later)

Michael #3: pystemd

  • Recall I recently build a Python-based systemd service for geo syncing my course materials
  • A thin Cython-based wrapper on top of libsystemd, focused on exposing the dbus API via sd-bus in an automated and easy to consume way.
  • By Alvaro Leiva, a production engineer at Facebook / Instagram
  • Presented at PyCon 2018
  • Systemd:
    • Manages your services and their lifetimes
    • e.g. I want my web app to start on boot but only after mongodb has started
  • pystemd lets you control and query these from a Python API

Brian #4: PyCharm 2018.2 EAP 1 includes improved pytest support

  • From Bruno Oliveira
    • “Oh my, full support for #pytest fixtures and parameterized tests coming in @pycharm 2018.2.“
  • “PyCharm 2018.2 supports using fixtures in Pytest. Using fixtures allows you to separate your setup code from the actual tests, making for more concise, and more readable tests. Additionally, there have been improvements to code navigation and refactoring Pytest tests, and to using parameterized tests.”
  • It’s hard for me to fully express how FREAKING EXCITED I am about this.
  • auto-complete now works with fixtures to test functions
  • goto declaration now works with fixtures to test functions
    • (not fixtures of fixtures, but they know about that already)
  • re-running a failed parametrization works (yay!)
  • re-running a single parametrization works (yay!)

Dan #5:

  • Why is installing Python 3.6 so hard? (Recent GitHub Bot workshop experience)
  • Sometimes hard to tell what’s easy/difficult for beginners
  • People hit crazy edge cases:
    • running Linux Subsystem for Windows (WSL) on Windows host, install Python into wrong environment
    • broken PPAs + bad StackOverflow advice → broken SSL and no pip on Ubuntu (deadsnakes PPA is the way to go)
    • People install multiple Python environments: Anaconda + python.org distribution
    • Hard to find instructions for compiling from source on Linux
  • Shameless plug: realpython.com/installing-python

Michael #6: 30 amazing Python projects (2018 edition)

  • Mybridge AI evaluates the quality by considering popularity, engagement and recency. To give you an idea about the quality, the average number of Github stars is 3,707.
  • No 30: PDFTabExtract: A set of tools for extracting tables from PDF files helping to do data mining on scanned documents.
  • No 28: Surprise v1.0: A Python scikit for building and analyzing recommender systems
  • No 27: Eel: A little Python library for making simple Electron-like HTML/JS GUI apps
  • No 25: Clairvoyant: A Python program that identifies and monitors historical cues for short term stock movement — Have you seen The Wall Street Code - VPRO documentary?
  • No 21: Fsociety: Hacking Tools Pack. A Penetration Testing Framework.
  • No 18: Maya: Datetime for Humans in Python
  • No 16: Better-exceptions: Pretty and useful exceptions in Python, automatically
  • No 13: Apistar: A fast and expressive API framework. For Python
  • No 8: MicroPython: A lean and efficient Python implementation for microcontrollers and constrained systems
  • No 6: spaCy (v2.0): Industrial-strength Natural Language Processing (NLP) with Python and Cython
  • No 2: Pytorch: Tensors and Dynamic neural networks in Python with strong GPU acceleration
  • No 1: Home-assistant (v0.6+): Open-source home automation platform running on Python 3

Our news

May 29 2018

30mins

Play

#96 Python Language Summit 2018

Podcast cover
Read more

Sponsored by DigitalOcean -- pythonbytes.fm/digitalocean

Brian #1: Plumbum: Shell Combinators and More

  • Toolbox of goodies to do shell-like things from Python.
  • “The motto of the library is “Never write shell scripts again”, and thus it attempts to mimic the shell syntax (shell combinators) where it makes sense, while keeping it all Pythonic and cross-platform.”

Example:

>>> from plumbum.cmd import grep, wc, cat, head
>>> chain = ls["-a"] | grep["-v", "\\.py"] | wc["-l"]
>>> print chain
/bin/ls -a | /bin/grep -v '\.py' | /usr/bin/wc -l
>>> chain()
u'13\n'
>>> ((cat < "setup.py") | head["-n", 4])()
u'#!/usr/bin/env python\nimport os\n\ntry:\n'
>>> (ls["-a"] > "file.list")()
u''
>>> (cat["file.list"] | wc["-l"])()
u'17\n'

Michael #2: Windows 10 Linux subsystem for Python developers

  • via Marcus Sherman
  • “One of the hardest days in teaching introduction to bioinformatics material is the first day: Setting up your machine.”
  • While I have seen a very large bias towards Macs in academia, there are plenty of people that keep their Windows machines as a badge of pride... Marcus included.
  • Even though Anaconda is cross platform and helpful, how does this work on Windows?
    • python3 -m venv .env and source .env/bin/activate?
    • Spoiler alert: Not well.
  • Step by step getting Ubuntu on Windows
  • Shows how to setup an x-server

Brian #3: Type hints cheat sheet (Python 3)

  • Do you remember how to type hint duck types?
    • Something accessed like an array (list or tuple or …) and holds strings → Sequence[str]
    • Something that works like a dictionary mapping integers to strings → Mapping[int, str]
  • As I’m adding more and more typing to interface functions, I keep this cheat sheet bookmarked.

Michael #4: Python driving new languages

  • Here are five predictions for what programming will look like 10 years from now.
    • Programming will be more abstract
    • Trends like serverless technologies, containers, and low code platforms suggest that many developers may work at higher levels of abstraction in the future
    • AI will become part of every developer's toolkit—but won't replace them
    • A universal programming language will arise
    • To reap the benefits of emerging technologies like AI, programming has to be easy to learn and easy to build upon
    • "Python may be remembered as being the great-great-great grandmother of languages of the future, which underneath the hood may look like the English language, but are far easier to use,"
    • Every developer will need to work with data
    • Programming will be a core tenet of the education system

Brian #5: asyncio documentation rewritten from scratch

  • twitter thread by Yury Selivanov
    • “Big news! asyncio documentation has been rewritten from scratch! Read the new version here: https://docs.python.org/3/library/asyncio.html ….
      Huge thanks to @WillingCarol, @elprans, and @andrew_svetlov for support, ideas, and reviews!’
    • “BTW, this is just the beginning. We'll continue to refine and update the documentation. Next up is adding two tutorials: one teaching high-level concepts and APIs, and another teaching how to use protocols and transports. A section about asyncio architecture is also planned.”
    • “And this is just the beginning not only for asyncio documentation, but for asyncio itself. Just for Python 3.8 we plan to add:
      • new streaming API
      • TaskGroups and cancel scopes
      • Supervisors and tracing API
      • new SSL implementation
      • many usability improvements”

Michael #6: The 2018 Python Language Summit

  • Here are the sessions:
    • Subinterpreter support for Python: a way to have a better story for multicore scalability using an existing feature of the language.
      • Subinterpreters will allow multiple Python interpreters per process and there is the potential for zero-copy data sharing between them.
      • But subinterpreters share the GIL, so that needs to be changed in order to make it multicore friendly.
    • Modifying the Python object model: looking at changes to CPython data structures to increase the performance of the interpreter.
      - via Instagram and Carl Shapiro
      - By modifying the Python object model fairly substantially, they were able to roughly double the performance
      - A little controversial
      - Shapiro's overall point was that he felt Python sacrificed its performance for flexibility and generality, but the dynamic features are typically not used heavily in performance-sensitive production workloads.
    • A Gilectomy update: a status report on the effort to remove the GIL from CPython.
      • Larry Hastings updated attendees on the status of his Gilectomy project.
      • Since his status report at last year's summit, little has happened, which is part of why the session was so short. He hasn't given up on the overall idea, but it needs a new approach.
    • Using GitHub Issues for Python: a discussion on moving from bugs.python.org to GitHub Issues.
    • Shortening the Python release schedule: a discussion on possibly changing from an 18-month to a yearly cadence.
      • The Python release cycle has an 18-month cadence; a new major release (e.g. Python 3.7) is made roughly on that schedule.
      • But Łukasz Langa, who is the release manager for Python 3.8 and 3.9, would like to see things move more quickly—perhaps on a yearly cadence.
    • Unplugging old batteries: should some older, unloved modules be removed from the standard library?
      • Python is famous for being a "batteries included" language—its standard library provides a versatile set of modules with the language
      • There may be times when some of those batteries have reached their end of life.
      • Christian Heimes wanted to suggest a few batteries that may have outlived their usefulness and to discuss how the process of retiring standard library modules should work.
    • Linux distributions and Python 2: the end of life for Python 2 is coming, what distributions are doing to prepare.
      • Christian Heimes wanted to suggest a few batteries that may have outlived their usefulness and to discuss how the process of retiring standard library modules should work.
      • To figure out how to help the Python downstreams so that Python 2 can be fully discontinued.
    • Python static typing update: a look at where static typing is now and where it is headed for Python 3.7.
      • Started things off by talking about stub files, which contain type information for libraries and other modules.
      • Right now, static typing is only partially useful for large projects because they tend to use a lot of packages from the Python Package Index (PyPI), which has limited stub coverage. There are only 35 stubs for third-party modules in the typeshed library, which is Python's stub repository.
      • He suggested that perhaps a centralized library for stubs is not the right development model. Some projects have stubs that live outside of typeshed, such as Django and SQLAlchemy.
      • PEP 561 ("Distributing and Packaging Type Information") will provide a way to pip install stubs from packages that advertise that they have them.
    • Python virtual environments: a short session on virtual environments and ideas for other ways to isolate local installations.
      • Steve Dower brought up the shortcomings of Python virtual environments, which are meant to create isolated installations of the language and its modules.
      • Thomas Wouters defended virtual environments in a response: The correct justification is that for the average person, not using a virtualenv all too soon creates confusion, pain, and very difficult to fix breakage. Starting with a virtualenv is the easiest way to avoid that, at very little cost.
      • But Beazley and others (including Dower) think that starting Python tutorials or training classes with a 20-minute digression on setting up a virtual environment is wasted time.
    • PEP 572 and decision-making in Python: a discussion of the controversy around PEP 572 and how to avoid the thread explosion that it caused in the future.
      • The "PEP 572 mess" was the topic of a 2018 Python Language Summit session led by benevolent dictator for life (BDFL) Guido van Rossum.
    • Getting along in the Python community: trying to find ways to keep the mailing list welcoming even in the face of rudeness.
      • About tkinter…
    • Mentoring and diversity for Python: a discussion on how to increase the diversity of the core development team.
      • Victor Stinner outlined some work he has been doing to mentor new developers on their path toward joining the core development ranks
      • Mariatta Wijaya gave a very personal talk that described the diversity problem while also providing some concrete action items that the project and individuals could take to help make Python more welcoming to minorities.

Extras

Listener feedback: CUDA is NVidia only, so no MacBook pro unless you have a custom external GPU.

Sep 22 2018

26mins

Play

#60 Don't dismiss SQLite as just a starter DB

Podcast cover
Read more

Brought to you by Datadog pythonbytes.fm/datadog

Brian #1: Who's at nine?

Michael #2: Retiring Python as a teaching language

  • Why did he write this?
    • Then one day a student will innocently ask "Instead of running the poker simulator from the command line, how can I put it in a window with a button to deal the next hand?"
  • The ensuing Twitter conversation was very interesting. Scroll this status, it’s pretty comprehensive https://twitter.com/mkennedy/status/949688651058835456

Brian #3: Don't dismiss SQLite as just a starter DB

  • SQLite is a single file db that comes with Python.
  • A listener pointed us to a couple cool things about SQLite
  • A great interview with the developer The Changelog, episode 201.
  • It's extensive documentation on how SQLite is tested.
  • Of course, for web applications and other applications that have to deal with extreme concurrency, you need a client server database
  • Many applications don't have extreme concurrency needs.
  • Sticking with SQLite might be just fine for quite a long time for many apps.

Michael #4: Chalice: Python Serverless Microframework for AWS

  • Chalice is a python serverless microframework for AWS. It allows you to quickly create and deploy applications that use Amazon API Gateway and AWS Lambda.
  • It provides:
    • A command line tool for creating, deploying, and managing your app
    • A familiar and easy to use API for declaring views in python code (Flask)
    • Automatic IAM policy generation
  • Compare to Zappa: https://github.com/Miserlou/Zappa

Brian #5: Fastest way to uniquely a list in Python >=3.6

  • Nice analysis of different ways to uniquify a list.
  • Punchline:
    • The fastest way to uniqify a list of hashable objects (basically immutable things) is: list(set(seq))
    • And the fastest way, if the order is important is: list(dict.fromkeys(seq))

Michael #6: PyTexas and PyCon AU vidoes are up

  • PyTexas
    • Notable PyTexas videos
      • Micropython
      • What is ML?
      • C for yourself
      • Python and .NET
  • PyCon AU
    • Notable PyCon AU videos
      • Gradual typing
      • Hot reloading Python web-servers at scale
      • Prototyping Python Microservices in Production
      • Secrets of a WSGI master.
      • Python 3 for People Who Haven't Been Paying Attention
      • Identity 2.0: the what, why and how of social and federated login
      • Python: Ludicrous mode (with Django)
      • Scaling Down: Running Large Sites Locally

Our news

Michael

Mastering PyCharm is out. Includes

  • Learn to manage Python projects in PyCharm (large and small)
  • Create web applications (Pyramid, Flask, Django, and more)
  • Use PyCharm's special data science mode
  • Refactor your Python code with confidence
  • Learn about code smells and duplicate code tooling
  • Access git, github, and use git flow
  • Use the visual debugger to understand code flow and state
  • Make your code more reliable with unit testing and pytest
  • Create new Python packages
  • And lots more

Webcast with JetBrains: MongoDB Quickstart with Python and PyCharm Jan 30

Jan 11 2018

26mins

Play

#132 Algorithms as objects

Podcast cover
Read more

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean

Brian #1: History of CircuitPython

  • PSF blog, A. Jesse Jiryu Davis
  • Adafruit hired Scott Shawcroft to port MicroPython to their SAMD21 chip they use on many of their boards.
  • CircuitPython is a friendly fork of MicroPython. Same licensing, and they share improvements back and forth.
  • “MicroPython customizes its hardware APIs for each chip family to provide speed and flexibility for hardware experts. Adafruit’s audience, however, is first-time coders. Shawcroft said, “Our goal is to focus on the first five minutes someone has ever coded.” “
  • “Shawcroft aims to remove all roadblocks for beginners to be productive with CircuitPython. As he demonstrated, CircuitPython auto-reloads and runs code when the user saves it; there are two more user experience improvements in the latest release. First, serial output is shown on a connected display, so a program like print("hello world") will have visible output even before the coder learns how to control LEDs or other observable effects.”
  • Related: CircuitPython 4.0.0 released

Michael #2: R Risks Python Swallowing It Whole: TIOBE

  • Is the R programming language in serious trouble? According to the latest update of the TIOBE Index, the answer seems to be “yes.”
  • R has finally tumbled out of the top 20 languages
  • “It seems that there is a consolidation going on in the statistical programming market. Python has become the big winner.”
  • Briefly speculates why is Python (which ranked fourth on this month’s list) winning big in data science? My thought: Python is a full spectrum language with solid numerical support.

Brian#3: The Missing Introduction To Containerization

  • Aymen El Amri
  • Understanding containerization through history
    • chroot jail, 1979, allowed isolation of a root process and it’s children from the rest of the OS, but with no security restrictions.
    • FreeBSD Jail, 2000, more secure, also isolating the file system.
    • Linux VServer, 2001, added “security contextes” and used new OS system-level virtualization. Allows you to run multiple Linux distros on a single VPS.
    • Oracle Solaris Containers, 2004, system resource controls and boundary separation provided by “zone”.
    • OpenVZ, 2005, OS-level virtualization. Used by many hosting companies to isolate and sell VPSs.
    • Google’s CGroups, 2007, a mechanizm to limit and isolate resource usage. Was mainlained into Linux kernel the same year.
    • LXC, Linux Containers, 2008, Similar to OpenVX, but uses CGroups.
    • CloudFoundry’s Warden, 2013, an API to manage environments.
    • Docker, 2013, os-level virtualization
    • Google’s LMCTFY (Let me contain that for you), 2014, an OSS version of Google’s container stack, providing Linux application containers. Most of this tech is being incorporated into libcontainer.
    • “Everything at Google runs on containers. There are more than 2 billion containers running on Google infrastructure every week.”
    • CoreOS’s rkt, 2014, an alternative to Docker.
  • Lots of terms defined
    • VPS, Virtual Machine, System VM, Process VM, …
  • OS Containers vs App Containers
  • Docker is both a Container and a Platform

  • This is halfway through the article, and where I got lost in an example on creating a container sort of from scratch. I think I’ll skip to a Docker tutorial now, but really appreciate the back story and mental model of containers.

Michael #4: Algorithms as objects

  • We usually think of an algorithm as a single function with inputs and outputs.
  • Our algorithms textbooks reinforce this notion.
  • They present very concise descriptions that neatly fit in half of a page.
  • Little details add up until you’re left with a gigantic, monolithic function
  • monolithic function lacks readability
  • the function also lacks maintainability
  • Nobody wants to touch this code because it’s such a pain to get any context
  • Complex code requires abstractions
  • How to tell if your algorithm is an object
  • Code smell #1. It’s too long or too deeply nested
  • Code smell #2. Banner comments
  • Code smell #3. Helper functions as nested closures, but it’s still too long
  • Code smell #4. There are actual helper functions, but they shouldn’t be called by anyone else
  • Code smell #5. You’re passing state between your helper functions
  • Write your algorithm as an object
  • Refactoring a monolithic algorithm into a class improves readability, which is is our #1 goal.
  • Lots of concrete examples in the article

Brian #5: pico-pytest

  • Oliver Bestwalter
  • Super tiny implementation of pytest core. 25 lines
  • My original hand crafted test framework was way more code than that, and not as readable.
  • This is good to look at to understand the heart of what test frameworks do
    • find test code
    • run it
    • mark any exceptions as failures
  • Of course, the bells and whistles added in the full implementation are super important, but this is the heart of what is happening.

Michael #6: An Introduction to Cython, the Secret Python Extension with Superpowers

  • Cython is one of the best kept secrets of Python.
  • It extends Python in a direction that addresses many of the shortcomings of the language and the platform, such as execution speed, GIL-free concurrency, absence of type checking and not creating an executable.
  • Number of widely used packages that are written in it, such as spaCy, uvloop, and significant parts of scikit-learn, Numpy and Pandas.
  • Cython makes use of the architectural organization of Python by translating (or 'transpiling', as it is now called) a Python file into the C equivalent of what the Python runtime would be doing, and compiling this into machine code.
  • Can sometimes avoid Python types altogether (e.g. sqrt function)
  • C arrays versus lists: Python collection types (list, dict, tuple and set) can be used as a type in cdef functions. The problem with the list structure, however, is that it leads to Python runtime interaction, and is accordingly slow
  • Nice article for getting started and motivation. But I didn’t see Python type annotations in play (they are now supported)

Extras

Brian:

Michael:

Jokes

  • What do you call eight hobbits? A hobbyte.
  • Two bytes meet. The first byte asks, 'Are you ill?' The second byte replies, 'No, just feeling a bit off.’
  • OR: What is Benoit B. Mandelbrot's middle name? Benoit B. Mandelbrot.

May 30 2019

30mins

Play

#67 Result of moving Python to Github

Podcast cover
Read more

Sponsored by DigitalOcean! http://do.co/python

Brian #1: Building a blog with Pelican

  • We did cover Pelican in episode 38, but this is a nice tutorial in 7 parts on building a blog.
  • Peter Kazarinoff, @pkazarinoff
    • Nice blog with a focus on engineering students.
  • Starts with installing Python and git and some other tools.
  • Step by step, every action to get a a blog up as a static site hosted on github pages.

Michael #2: Notifiers

  • Got an app or service and you want to enable your users to use notifications with their provider of choice?
  • Working on a script and you want to receive notification based on its output?
  • A one stop shop for all notification providers with a unified and simple interface.
  • A unified interface means that you already support any new providers that will be added, no more work needed!
  • Some providers
    • Slack
    • Gmail
    • Telegram
    • Gitter
  • Python 3 only

Brian #3: Using Makefiles in Python projects

  • Krzysztof Żuraw, @krzysztof_zuraw
  • Alerted to this article from kidpixo, @kidpixo
  • We don’t usually think of Makefiles and Python together, but they can be a handy place to keep common scripts for a project all in one place.
  • This article is a nice gentle intro to Makefiles and shows some cool uses:
    • cleaning out .pyc files
    • cleaning out egg directories
    • linting
    • running tests with flags
    • starting a test server
    • deploying
    • sorting import files

Michael #4: Result of moving Python to Github

  • See the graph linked in the post
  • A couple of quick numbers (including PRs too) from 2017 compared to 2016:
    • the number of commit has increased by 190%
    • inserted lines of code has increased by 140%
    • number of unique contributors has increased by 1300%
    • number of returning contributors has increased by 900%
  • One comment was: “Personally, I would like them moving to Gitlab instead, but still good news.” I tend to disagree.

Brian #5: Self-Deprecation Needs to Stop

  • Maurice Hayward, @maurice_hayward
  • Inspired by some tweets by Stephanie Hurlburt, @sehurlburt
  • Stop saying these words when describing yourself or your accomplishments.
    These words are now under BAN:
    "My project is..."
    - very small/basic/simple
    - not that good
    - a thing I wrote
    - just by a newbie
    - something I didn't spend a lot of time/effort on
    - silly
    - not that useful
    Just state the topic and let others be the judge.
  • Really think about the value you bring, then let everybody know.
  • Be Proud of Yourself!

Michael #6: 5 speed improvements in Python 3.7

  1. Calling methods faster (maybe)
    1. Python 3.7 adds 2 new Opcodes, LOAD_METHOD and CALL_METHOD for when the compiler sees x.method(...) it uses these new Opcodes.
    2. Bound methods with no arguments are now faster
  2. str.find() is faster for some characters
    1. Some unicode characters have an unfortunate issue when scanning a string for occurrences using str.find(x), seeing up to 25x slow down.
    2. These are still slower, but now 3x slower than ASCII characters instead of 25x!
  3. os.fwalk is 2x faster
  4. Regular expressions are faster
    1. A change was made in Python 3.6 which slowed down this call when flags were passed which were integers. Python 3.7 “fixes” the slowdown but is still not as fast as Python 3.5
  5. Regular expressions are faster for case-insensitive matching
    1. The speed improvement is significant, if you’re matching ASCII characters you can see up to a 20x improvements in matching time since it’s now doing a lookup instead of running lower() over each character.

Follow up and other news

Brian:

Mar 01 2018

21mins

Play

#70 Have you seen my log? It's cute!

Podcast cover
Read more

Sponsored by DigitalOcean: do.co/python

Brian #1: Online CookieCutter Generator

  • “Get a ZIP-archive with project by filling out the form.”
  • By @kpavlovsky_pro Konstantin Pavlovsky

Michael #2: cutelog – GUI for Python's logging module

  • This is a graphical log viewer for Python's standard logging module.
  • Features
    • Allows any number of simultaneous connections
    • Fully customizable look of log levels and columns
    • Filtering based on level and name of the logger, as well as filtering by searching
    • Search through all records or only through filtered ones
    • View exception tracebacks or messages in a separate window
    • Dark theme (with its own set of colors for levels)
    • Pop tabs out of the window, merge records of multiple tabs into one
  • Based on PyQt5 speaking of GUIs

Brian #3: wagtail 2.0

  • “Wagtail is a content management system built on Django. It’s focused on user experience, and offers precise control for designers and developers.”
  • The Zen of Wagtail - nice philosophy of the project page to let you know if this kind of thing is right for you and your project.
  • In 2.0
    • a new text editor
    • Django 2 support
    • better scheduled publishing
  • wagtail docs
  • gallery of sites made with wagtail

Michael #4: peewee 3.0 is out

  • Peewee is a simple and small ORM. It has few (but expressive) concepts, making it easy to learn and intuitive to use.
    • A small, expressive ORM
    • Written in python with support for versions 2.7+ and 3.4+ (developed with 3.6)
    • Built-in support for SQLite, MySQL and Postgresql.
    • Numerous extensions available (postgres hstore/json/arrays, sqlite full-text-search, schema migrations, and much more).
  • Although this was pretty much a complete rewrite of the 2.x codebase, I have tried to maintain backwards-compatibility for the public APIs.
  • Exciting because of its async support via peewee-async
    • peewee-async is a library providing asynchronous interface powered by asyncio for peewee ORM.
database.set_allow_sync(False)

async def handler():
await objects.create(TestModel, text="Not bad. Watch this, I'm async!")
all_objects = await objects.execute(TestModel.select())
for obj in all_objects:
print(obj.text)

Brian #5: Machine Learning Basics

  • “Plain python implementations of basic machine learning algorithms”
  • From the repo:
    • A repository of implementations of basic machine learning algorithms in plain Python (Python Version 3.6+). All algorithms are implemented from scratch without using additional machine learning libraries. The intention of these notebooks is to provide a basic understanding of the algorithms and their underlying structure, not to provide the most efficient implementations.
      • Linear Regression
      • Logistic Regression
      • Perceptron
      • k-nearest-neighbor
      • k-Means clustering
      • Simple neural network with one hidden layer
      • Multinomial Logistic Regression

Michael #6: Cerberus

  • Cerberus provides powerful yet simple and lightweight data validation functionality out of the box
  • designed to be easily extensible, allowing for custom validation
  • Origin of the name: CERBERUS, n. The watch-dog of Hades, whose duty it was to guard the entrance;
schema = {'name': {'type': 'string'}, 'age': {'type': 'integer', 'min': 10}}
v = Validator(schema)

document = {'name': 'Little Joe', 'age': 5}
v.validate(document) # False


v.errors # {'age': ['min value is 10']}

Follow up and other news

Michael:

#100DaysOfCode in Python course: talkpython.fm/100days

Mar 23 2018

15mins

Play

#159 Brian's PR is merged, the src will flow

Podcast cover
Read more

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean

Michael #1: Final type

  • PEP 591 -- Adding a final qualifier to typing
  • This PEP proposes a "final" qualifier to be added to the typing module---in the form of a final decorator and a Final type annotation---to serve three related purposes:
    • Declaring that a method should not be overridden
    • Declaring that a class should not be subclassed
    • Declaring that a variable or attribute should not be reassigned
  • Some situations where a final class or method may be useful include:
    • A class wasn’t designed to be subclassed or a method wasn't designed to be overridden. Perhaps it would not work as expected, or be error-prone.
    • Subclassing or overriding would make code harder to understand or maintain. For example, you may want to prevent unnecessarily tight coupling between base classes and subclasses.
    • You want to retain the freedom to arbitrarily change the class implementation in the future, and these changes might break subclasses.
# Example for a class:
from typing import final

@final
class Base:
...

class Derived(Base): # Error: Cannot inherit from final class "Base"
...

And for a method:

class Base:
@final
def foo(self) -> None:
...

class Derived(Base):
def foo(self) -> None: # Error: Cannot override final attribute "foo"
# (previously declared in base class "Base")
...
  • It seems to also mean const
RATE: Final = 3000

class Base:

DEFAULT_ID: Final = 0

RATE = 300 # Error: can't assign to final attribute
Base.DEFAULT_ID = 1 # Error: can't override a final attribute

Brian #2: flit 2

Michael #3: Pint

  • via Andrew Simon
  • Physical units and builtin unit conversion to everyday python numbers like floats.
  • Receive inputs in different unit systems it can make life difficult to account for that in software.
  • Pint handles the unit conversion automatically in a wide array of contexts – Can add 2 meters and 5 inches and get the correct result without any additional work.
  • The integration with numpy and pandas are seamless, and it’s made my life so much simpler overall.
  • Units and types of measurements
  • Think you need this? How about the Mars Climate Orbiter
    • The MCO MIB has determined that the root cause for the loss of the MCO spacecraft was the failure to use metric units in the coding of a ground software file, “Small Forces,” used in trajectory models. Specifically, thruster performance data in English units instead of metric units was used in the software application code titled SM_FORCES (small forces).

Brian #4: 8 great pytest plugins

  • Jeff Triplett

Michael #5: 11 new web frameworks

  1. Sanic [flask like] - a web server and web framework that’s written to go fast. It allows the usage of the async / await syntax added in Python 3.5
  2. Starlette [flask like] - A lightweight ASGI framework which is ideal for building high performance asyncio services, designed to be used either as a complete framework, or as an ASGI toolkit.
  3. Masonite - A developer centric Python web framework that strives for an actual batteries included developer tool with a lot of out of the box functionality. Craft CLI is the edge here.
  4. FastAPI - A modern, high-performance, web framework for building APIs with Python 3.6+ based on standard Python type hints.
  5. Responder - Based on Starlette, Responder’s primary concept is to bring the niceties that are brought forth from both Flask and Falcon and unify them into a single framework.
  6. Molten - A minimal, extensible, fast and productive framework for building HTTP APIs with Python. Molten can automatically validate requests according to predefined schemas.
  7. Japronto - A screaming-fast, scalable, asynchronous Python 3.5+ HTTP toolkit integrated with pipelining HTTP server based on uvloop and picohttpparser.
  8. Klein [flask like] - A micro-framework for developing production-ready web services with Python. It is ‘micro’ in that it has an incredibly small API similar to Bottle and Flask.
  9. Quart [flask like]- A Python ASGI web microframework. It is intended to provide the easiest way to use asyncio functionality in a web context, especially with existing Flask apps.
  10. BlackSheep - An asynchronous web framework to build event based, non-blocking Python web applications. It is inspired by Flask and ASP.NET Core. BlackSheep supports automatic binding of values for request handlers, by type annotation or by conventions.
  11. Cyclone - A web server framework that implements the Tornado API as a Twisted protocol. The idea is to bridge Tornado’s elegant and straightforward API to Twisted’s Event-Loop, enabling a vast number of supported protocols.

Brian #6: Raise Better Exceptions in Python

Extras

Michael:

Joke

  • via Daniel Pope
  • What's a tractor's least favorite programming language? Rust.

Dec 03 2019

33mins

Play

#158 There's a bounty on your open-source bugs!

Podcast cover
Read more

This episode is sponsored by DigitalOcean - pythonbytes.fm/digitalocean

Brian #1: Python already replaced Excel in banking

  • “If you wanted to prove your mettle as an entry-level banker or trader it used to be the case that you had to know all about financial modeling in Excel. Not any more. These days it's all about Python, especially on the trading floor.
  • "Python already replaced Excel," said Matthew Hampson, deputy chief digital officer at Nomura, speaking at last Friday's Quant Conference in London. "You can already walk across the trading floor and see people writing Python code...it will become much more common in the next three to four years."

Michael #2: GitHub launches 'Security Lab' to help secure open source ecosystem

  • At the GitHub Universe developer conference, GitHub announced the launch of a new community program called Security Lab
  • GitHub says Security Lab founding members have found, reported, and helped fix more than 100 security flaws already.
  • Other organizations, as well as individual security researchers, can also join. A bug bounty program with rewards of up to $3,000 is also available, to compensate bug hunters for the time they put into searching for vulnerabilities in open source projects.
  • Bug reports must contain a CodeQL query. CodeQL is a new open source tool that GitHub released today; a semantic code analysis engine that was designed to find different versions of the same vulnerability across vasts swaths of code.
  • Starting today automated security updates are generally available and have been rolled out to every active repository with security alerts enabled.
  • Once a security flaw is fixed, the project owner can publish the security, and GitHub will warn all upstream project owners who are using vulnerable versions of the original maintainer's code.
  • But before publishing a security advisory, project owners can also request and receive a CVE number for their project's vulnerability directly from GitHub.
  • And last, but not least, GitHub also updated Token Scanning, its in-house service that can scan users' projects for API keys and tokens that have been accidentally left inside their source code.

Brian #3: pybit.es now has some test challenges

Michael #4: pyhttptest - a command-line tool for HTTP tests over RESTful APIs

  • via Florian Dahlitz
  • A command-line tool for HTTP tests over RESTful APIs
  • Tired of writing test scripts against your RESTFul APIs anytime? Describe an HTTP Requests test cases in a simplest and widely used format JSON within a file. Run one command and gain a summary report.
  • Example
{
"name": "TEST: List all users",
"verb": "GET",
"endpoint": "users",
"host": "https://github.com",
"headers": {
"Accept-Language": "en-US"
},
"query_string": {
"limit": 5
}
}

Brian #5: xarray

  • suggested by Guido Imperiale
  • xarray is a mature library that builds on top of numpy, pandas and dask to offer arrays that are
    • n-dimensional (numpy and dask do it, but pandas doesn't)
    • self-described and indexed (pandas does it, but numpy and dask don't)
    • out-of-memory, multi-threaded, and cloud-distributed (dask does it, but numpy and pandas don't).
  • Additionally, xarray can semi-transparently swap numpy with other backends, such as sparse , while retaining the same API.

Michael #6: Animated SVG Terminals

  • Florian Dahlitz
  • termtosvg is a Unix terminal recorder written in Python that renders your command line sessions as standalone SVG animations.
  • Features:
    • Produce lightweight and clean looking animations or still frames embeddable on a project page
    • Custom color themes, terminal UI and animation controls via user-defined SVG templates
    • Rendering of recordings in asciicast format made with asciinema
  • Examples: nbedos.github.io/termtosvg/pages/examples.html

Extras

  • pytest 5.3.0 released, please read changelog if you use pytest, especially if you use it with CI and depend on --junitxml, as they have changed the format to a newer version.

Michael:

Jokes

  • What do you get when you put root beer in a square glass? Beer.

  • Q: What do you call optimistic front-end developers?

  • A: Stack half-full developers.

  • Also, I was going to tell a version control joke, but they are only funny if you git them.

Nov 27 2019

26mins

Play

#157 Oh hai Pandas, hold my hand?

Podcast cover
Read more

This episode is sponsored by DigitalOcean: pythonbytes.fm/digitalocean

Michael #1: pydantic

  • via Colin Sullivan
  • Data validation and settings management using python type annotations.
  • (We covered Cerberus, this is similar)
  • pydantic enforces type hints at runtime, and provides user friendly errors when data is invalid.
class User(pydantic.BaseModel):
id: int
name = 'John Doe'
signup_ts: datetime = None
friends: List[int] = []

external_data = {
'id': '123',
'signup_ts': '2019-06-01 12:22',
'friends': [1, 2, '3']
}
user = User(**external_data)
  • id is of type int; the annotation-only declaration tells pydantic that this field is required. Strings, bytes or floats will be coerced to ints if possible; otherwise an exception will be raised.
  • name is inferred as a string from the provided default; because it has a default, it is not required.
  • signup_ts is a datetime field which is not required (and takes the value None if it's not supplied).
  • Why use it?

Brian #2: Coverage.py 5.0 beta 1 adds context support

  • Please try out the beta, even without trying contexts, as it helps Ned Batchelder to make sure it’s as backwards compatible as possible while still adding this super cool functionality.
  • Trying out contexts with pytest and pytest-cov:
(venv) $ pip install coverage==5.0b1
(venv) $ pip install pytest-cov
(venv) $ pytest --cov=foo --cov-context=test test_foo.py
(venv) $ coverage html --show-contexts
(venv) $ open htmlcov/index.html
  • results in coverage report that has little dropdowns on the right for lines that are covered, and what context they were covered.
  • For the example above, with pytest-cov, it shows what test caused each line to be hit.
  • Contexts can do way more than this. One example, split up different levels of tests, to see which lines are only hit by unit tests, indicating missing higher level tests, or the opposite.
  • The stored db could also possibly be mined to see how much overlap there is between tests, and maybe help with higher level tools to predict the harm or benefit from removing some tests.
  • I’m excited about the future, with contexts in place.
  • Even if you ignore contexts, please go try out the beta ASAP to make sure your old use model still works.

Michael #3: PSF is seeking developers for paid contract improving pip

  • via Brian Rutledge
  • The Python Software Foundation Packaging Working Group is receiving funding to work on the design, implementation, and rollout of pip's next-generation dependency resolver.
  • This project aims to complete the design, implementation, and rollout of pip's next-generation dependency resolver.
  • Lower the barriers to installing Python software, empowering users to get a version of a package that works.
  • It will also lower the barriers to distributing Python software, empowering developers to make their work available in an easily reusable form.
  • Because of the size of the project, funding has been allocated to secure two contractors, a senior developer and an intermediate developer, to work on development, testing and building test infrastructure, code review, bug triage, and assisting in the rollout of necessary features.
  • Total pay: Stage 1: $116,375, Stage 2: $103,700

Brian #4: dovpanda - Directions OVer PANDAs

  • Dean Langsam
  • “Directions are hints and tips for using pandas in an analysis environment. dovpanda is an overlay for working with pandas in an analysis environment.
  • "If you think your task is common enough, it probably is, and Pandas probably has a built-in solution. dovpanda is an overlay module that tries to understand what you are trying to do with your data, and help you find easier ways to write your code.”
  • “The main usage of dovpanda is its hints mechanism, which is very easy and works out-of-the-box. Just import it after you import pandas, whether inside a notebook or in a console.”
  • It’s like training wheels for pandas to help you get the most out of pandas and learn while you are doing your work. Very cool.

Michael #5: removestar

  • via PyCoders newsletter
  • Tool to automatically replace 'import *' in Python files with explicit imports
  • Report only mode and modify in place mode.

Brian #6: pytest-quarantine : Save the list of failing tests, so that they can be automatically marked as expected failures on future test runs.

  • Brian Rutlage
  • Really nice email from Brian:
  • >"Hi Brian! We've met a couple times at PyCon in Cleveland. Thanks for your podcasts, and your book. I've gone from being a complete pytest newbie, to helping my company adopt it, to writing a plugin. The plugin was something I developed at work, and they let me open-source it. I wanted to share it with you as a way of saying "thank you", and because you seem to be a bit of connoisseur of pytest plugins. ;)"
  • Here it is: https://github.com/EnergySage/pytest-quarantine/
  • pytest has a cool feature called xfail, to allow you to mark tests you know fail.
  • pytest-quarantine allows you to run your suite and generate a file of all failures, then use that to mark the xfails.
    • Then you or your team can chip away at these failures until you get rid of them.
    • But in the meantime, your suite can still be useful for finding new failures.
    • And, the use of an external file to mark failures makes it so you don’t have to edit your test files to mark the tests that are xfail.

Extras:

MK: Our infrastructure is fully carbon neutral!

Joke:

A cop pulls Dr. Heisenberg over for speeding. The officer asks, "Do you know how fast you were going?" Heisenberg pauses for a moment, then answers, "No, but I know where I am.” [1]

  1. See Uncertainty principle, also called Heisenberg uncertainty principle or indeterminacy principle, statement, articulated (1927) by the German physicist Werner Heisenberg, that the position and the velocity of an object cannot both be measured exactly, at the same time, even in theory.

Nov 20 2019

23mins

Play

#156 All the programming LOLs

Podcast cover
Read more

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean

Special guests:

Dan #1: Why You Should Use python -m pip

Cecil #2: Visual Studio Online: Web-Based IDE & Collaborative Code Editor

Michael #3: Python Adopts a 12-month Release Cycle

  • The long discussion on changing the Python project's release cadence has come to a conclusion: the project will now be releasing new versions on an annual basis.
  • Described in PEP 602
  • The steering council thinks that having a consistent schedule every year when
  • we hit beta, RC, and final it will help the community:
    • Know when to start testing the beta to provide feedback
    • Known when the expect the RC so the community can prepare their projects for the final release
    • Know when the final release will occur to coordinate their own releases (if necessary) when the final release of Python occurs
    • Allow core developers to more easily plan their work to make sure work lands in the release they are targeting
    • Make sure that core developers and the community have a shorter amount of time to wait for new features to be released

Dan #4: Black 19.10b0 Released — stable release coming soon

Cecil 5: Navigating code on GitHub

Michael #6: lolcommits: selfies for software developers.

  • lolcommits takes a snapshot with your webcam every time you git commit code, and archives a lolcat style image with it. git blame has never been so much fun.
  • Infinite uses: Animate your progress through a project and watch as you age. See what you looked like when you broke the build. Keep a joint lolrepository for your entire company.
  • Plugins: Lolcommits allows a growing list of plugins to perform additional work on your lolcommit image after capturing.
  • Animate: Configure lolcommits to generate an animated GIF with each commit for extra lulz!

Extras:

Dan:

Cecil:

Michael:

Joke:

LOLCODE

Nov 15 2019

28mins

Play

#155 Guido van Rossum retires

Podcast cover
Read more

Sponsored by Datadog: pythonbytes.fm/datadog

Michael #1: Guido retires

  • Guido van Rossum has left DropBox and retired (post)
  • Let’s all take a moment to say thank you.
  • I wonder what will come next in terms of creative projects
  • Some comments from community members (see Twitter thread)

Brian #2: SeleniumBase

  • Automated UI Testing with Selenium WebDriver and pytest.
  • Very expressive and intuitive automation library built on top of Selenium WebDriver. method overview
  • very readable (this is a workflow test, but still, quite readable):
    from seleniumbase import BaseCase
class MyTestClass(BaseCase):
def test_basic(self):
self.open("https://xkcd.com/353/")
self.assert_title("xkcd: Python")
self.assert_element('img[alt="Python"]')
self.click('a[rel="license"]')
self.assert_text("free to copy and reuse")
self.go_back()
self.click("link=About")
self.assert_text("xkcd.com", "h2")
self.open("https://store.xkcd.com/collections/everything")
self.update_text("input.search-input", "xkcd book\n")
self.assert_exact_text("xkcd: volume 0", "h3")
  • includes plugins for including screenshots in test results.
  • supports major CI systems
  • some cool features that I didn’t expect
    • user onboarding demos
    • assisted QA (partially automated with manual questions)
    • support for selenium grid
    • logs of command line options, including headless

Michael #3: Reimplementing a Solaris command in Python gained 17x performance improvement from C

  • Postmortem by Darren Moffat
  • Is Python slow?
  • A result of fixing a memory allocation issue in the /usr/bin/listusers command
  • Decided to investigate if this ancient C code could be improved by conversion to Python.
  • The C code was largely untouched since 1988 and was around 800 lines long, it was written in an era when the number of users was fairly small and probably existed in the local files /etc/passwd or a smallish NIS server.
  • It turns out that the algorithm to implement the listusers is basically some simple set manipulation.
  • Rewrite of listusers in Python 3 turned out to be roughly a 10th of the number of lines of code
  • But Python would be slower right ? Turns out it isn't and in fact for some of my datasets (that had over 100,000 users in them) it was 17 times faster.
  • A few of the comments asked about the availability of the Python version. The listusers command in Oracle Solaris 11.4 SRU 9 and higher. Since we ship the /usr/bin/listusers file as the Python code you can see it by just viewing the file in an editor. Note though that is not open source and is covered by the Oracle Solaris licenses.

Brian #4: 20 useful Python tips and tricks you should know

  • I admit it, I’m capable of getting link-baited by the occasional listicle.
  • Some great stuff, especially for people coming from other languages.
    • Chained assignment: x = y = z = 2
    • Chained comparison:
      • 2 < x <= 8
      • 2 < x > 4
      • 0 < x < 4 < y < 16
  • Multiple assignment: x, y, z = 2, 4, 8
  • More Advanced Multiple Assignment: x, *y, z = 2, 4, 8, 16
    • I’ve been using the * for unpacking a lot, especially with *_
  • Merge dictionaries: z = {**x, **y}
  • Join strings: '_'.join(['u', 'v', 'w'])
  • using list(set(something)) to remove duplicates.
  • aggregate elements. using zip to element-wise combine two or more iterables.
>>> x = [1, 2, 3]
>>> y = ['a', 'b', 'c']
>>> zip(x, y)
[(1, 'a'), (2, 'b'), (3, 'c')]
  • and then some other weird stuff that I don’t find that useful.

Michael #5: Complexity Waterfall

  • via Ahrem Ahreff
  • Heavy use of wemake-python-styleguide
  • Code smells!
  • Use your refactoring tools and write tests.
  • Automation enable an opportunity of “Continuous Refactoring” and “Architecture on Demand” development styles.

Brian #6: Plynth

  • Plynth is a GUI framework for building cross-platform desktop applications with HTML, CSS and Python.
  • Plynth has integrated the standard CPython implementation with Chromium's rendering engine. You can run your python scripts seamlessly with HTML/CSS instead of using Javascript with modules from pip
  • Plynth uses Chromium/Electron for its rendering. With Plynth, every Javascript library turns into a Python module.
  • Not open source. But free for individuals, including commercial use and education.
  • A bunch of tutorial videos that are not difficult to follow, and not long, but… not really obvious code either.
  • Python 3.6 and 3.7 development kits available

Extras:

Michael:

Joke:

  • Q: What's a web developer's favorite tea?
  • A: URL gray
  • via Aideen Barry

Nov 06 2019

32mins

Play

#154 Code, frozen in carbon, on display for all

Podcast cover
Read more

Sponsored by Datadog: pythonbytes.fm/datadog

Special guest: Bob Belderbos

Brian #1: Lesser Known Coding Fonts

  • Interesting examination of some coding fonts.
  • Link to a great talk called Cracking the Code, by Jonathan David Ross, about coding fonts and Input.

I’m trying out Input Mono right now, and quite like it.

Bob #2: Django Admin Handbook

Michael #3: Your Guide to the CPython Source Code

  • Let’s talk about exploring the CPython code
  • You’ll want to get the code: git clone https://github.com/python/cpython
  • Compile the code (Anthony gives lots of steps for macOS, Windows, and Linux)
  • Structure:
cpython/

├── Doc ← Source for the documentation
├── Grammar ← The computer-readable language definition
├── Include ← The C header files
├── Lib ← Standard library modules written in Python
├── Mac ← macOS support files
├── Misc ← Miscellaneous files
├── Modules ← Standard Library Modules written in C
├── Objects ← Core types and the object model
├── Parser ← The Python parser source code
├── PC ← Windows build support files
├── PCbuild ← Windows build support files for older Windows versions
├── Programs ← Source code for the python executable and other binaries
├── Python ← The CPython interpreter source code
└── Tools ← Standalone tools useful for building or extending Python
  • Some cool “hidden” goodies. For example, check out Lib/concurrent/futures/process.py, it comes with a cool ascii diagram of the process.
  • Lots more covered, that we don’t have time for
    • The Python Interpreter Process
    • The CPython Compiler and Execution Loop
    • Objects in CPython
    • The CPython Standard Library
    • Installing a custom version

Brian #4: Six Django template tags not often used in tutorials

  • Here’s a few:
    • {% empty %}, for use in for loops when the array is empty
    • {% lorem \[count\] [method] [random] %} for automatically filling with Lorem Ipsum text.
    • {% verbatim %} … {% endverbatim %}, stop the rendering engine from trying to parse it and replace stuff.
    • https://hipsum.co/

Bob #5: Beautiful code snippets with Carbon

  • Beautiful images, great for teaching Python / programming.
  • Used by a lot of developer, nice example I spotted today.
  • Supports typing and drag and drop, just generated this link by dropping a test module onto the canvas!
  • Great to expand Twitter char limit (we use it to generate Python Tip images).
  • Follow the project here, seems they now integrate with Github.

Michael #6: Researchers find bug in Python script may have affected hundreds of studies

  • More info via Mike Driscoll at Thousands of Scientific Papers May be Invalid Due to Misunderstanding Python
  • In a paper published October 8, researchers at the University of Hawaii found that a programming error in a set of Python scripts commonly used for computational analysis of chemistry data returned varying results based on which operating system they were run on.
  • Scientists did not understand that Python’s glob.glob() does not return sorted results
  • Throwing doubt on the results of more than 150 published chemistry studies.
  • the researcher were trying to analyze results from an experiment involving cyanobacteria discovered significant variations in results run against the same nuclear magnetic resonance spectroscopy (NMR) data.
  • The scripts, called the "Willoughby-Hoye" scripts after their creators, were found to return correct results on macOS Mavericks and Windows 10. But on macOS Mojave and Ubuntu, the results were off by nearly a full percent.
  • The module depends on the operating system for the order in which the files are returned. And the results of the scripts' calculations are affected by the order in which the files are processed.
  • The fix: A simple list.sort()!
  • Williams said he hopes the paper will get scientists to pay more attention to the computational side of experiments in the future.

Extras:

Working on: PyBites platform: added flake8/ black code formatting, UI enhancements.

Michael:

Joke:

  • Q: What did the Network Administrator say when they caught a nasty virus?
  • A: It hurts when IP

Oct 29 2019

32mins

Play

#153 Auto format my Python please!

Podcast cover
Read more

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean

Michael #1: Building a Python C Extension Module

  • Tutorial, learn to use the Python API to write Python C extension modules.
  • And
    • Invoke C functions from within Python
    • Pass arguments from Python to C and parse them accordingly
    • Raise exceptions from C code and create custom Python exceptions in C
    • Define global constants in C and make them accessible in Python
    • Test, package, and distribute your Python C extension module
  • Extending Your Python Program
    • there may be other lesser-used system calls that are only accessible through C
  • Steps: Writing a Python Interface in C
    • Figure out the arguments (e.g. int fputs(const char *, FILE *) )
    • Implement in C:
#include Python.h
static PyObject *method_fputs(PyObject *self, PyObject *args) {
char *str, *filename = NULL;
int bytes_copied = -1;
/* Parse arguments */
if(!PyArg_ParseTuple(args, "ss", &str, &filename)) {
return NULL;
}
FILE *fp = fopen(filename, "w");
bytes_copied = fputs(str, fp);
fclose(fp);
return PyLong_FromLong(bytes_copied);
}
  • In line 2, you declare the argument types you wish to receive from your Python code
  • line 6, then you’ll see that PyArg_ParseTuple() copies into the char*’s
  • Write regular C code (fopen, fputs)
  • Return: PyLong_FromLong() creates a PyLongObject, which represents an integer object in Python.
  • a few extra functions that are necessary
    • write definitions of your module and the methods it contains
    • Before you can import your new module, you first need to build it. You can do this by using the Python package distutils.

Brian #2: What’s New in Python 3.8 - docs.python.org

We’ve already talked about the big hitters:

  • assignment expressions, (the walrus operator)
  • positional only parameters, (the / in the param list)
  • f-strings support = for self-documenting expressions and debugging

There are a few more goodies I wanted to quickly mention:

  • More async: python -m asyncio launches a native async REPL
  • More helpful warnings and messages when
    • using is and is not to compare strings and integers and other types intended to be compared with == and !=
    • Missing the comma in stuff like [(1,2) (3,4)]. Happens all the time with parametrized testing
  • you can do iterable unpacking in a yield or return statement
    • x = (1, 2, 3)
    • a, *b = x
    • return a, *b <- this used to be a syntax error
      • you had to do return (a, *b)
  • New module importlib.metadata lets you access things like version numbers or dependent library required version numbers, and cool stuff like that.

Michael #3: UK National Cyber Security Centre (NCSC) is warning developers of the risks of sticking with Python 2.7, particularly for library writers

  • NCSC likens companies continuing to use Python 2 past its EOL to tempting another WannaCry or Equifax incident.
    • Equifax details: a vulnerability, dubbed CVE-2017-5638, was discovered in Apache Struts, an open source development framework for creating enterprise Java applications that Equifax, along with thousands of other websites, uses…
  • Quote: "If you're still using 2.x, it's time to port your code to Python 3," the NCSC said. "If you continue to use unsupported modules, you are risking the security of your organisation and data, as vulnerabilities will sooner or later appear which nobody is fixing."
  • Moreover: "If you maintain a library that other developers depend on, you may be preventing them from updating to 3," the agency added. "By holding other developers back, you are indirectly and likely unintentionally increasing the security risks of others.”
  • "If migrating your code base to Python 3 is not possible, another option is to pay a commercial company to support Python 2 for you," the NCSC said.
  • NCSC: If you don't migrate, you should expect security incidents
  • Python's popularity makes updating code imperative: The reason the NCSC is warning companies about Python 2's impending EOL is because of the language's success.

Brian #4: Pythonic News

  • Sebastian A. Steins
  • “A Hacker News lookalike written in Python/Django”
  • “ powering https://news.python.sc"
  • Cool that it’s open source, and on github
  • Was submitted to us by Sebastian, and a few others too, so there is excitement.
  • It’s like 6 days old and has 153 stars on github, 4 contributors, 18 forks.
  • Fun.

Michael #5: Deep Learning Workstations, Servers, Laptops, and GPU Cloud

  • GPU-accelerated with TensorFlow, PyTorch, Keras, and more pre-installed. Just plug in and start training. Save up to 90% by moving off your current cloud and choosing Lambda.
  • They offer:
    • TensorBook: GPU Laptop for $2,934
    • Lambda Quad: 4x GPU Workstation for $21,108 (yikes!)
    • All in: Lambda Hyperplane: 8x Tesla V100 Server, starting at $114,274
  • But compare to:
    • AWS EC2: p3.8xlarge @ $12.24 per Hour => $8,935 / month

Brian #6: Auto formatters for Python

  • A comparison of autopep8, yapf, and black
  • Auto formatters are super helpful for teams. They shut down the unproductive arguments over style and make code reviews way more pleasant. People can focus on content, not being the style police.
  • We love black. But it might be a bit over the top for some people. Here are a couple of other alternatives.
  • autopep8 - mostly focuses on PEP8
    • “autopep8 automatically formats Python code to conform to the PEP 8 style guide. It uses the pycodestyle utility to determine what parts of the code needs to be formatted. autopep8 is capable of fixing most of the formatting issues that can be reported by pycodestyle.”
  • black - does more
    • doesn’t have many options, but you can alter line length, can turn of string quote normalization, and you can limit or focus the files it sees.
    • does a cool “check that the reformatted code still produces a valid AST that is equivalent to the original.” but you can turn that off with --fast
  • yapf - way more customizable.
    • Great if you want to auto format to a custom style guide.
    • “The ultimate goal is that the code YAPF produces is as good as the code that a programmer would write if they were following the style guide. It takes away some of the drudgery of maintaining your code.”
  • Article is cool in that it shows some sample code and how it’s changed by the different formatters.

Extras:

Michael:

  • New courses coming
  • Financial Aid Launches for PyCon US 2020!

Joke:

  • American Py Song

  • From Eric Nelson:

    • Math joke. “i is as complex as it gets. jk.”

Oct 23 2019

26mins

Play

#152 You have 35 million lines of Python 2, now what?

Podcast cover
Read more

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean

Michael #1: JPMorgan’s Athena Has 35 Million Lines of Python 2 Code, and Won’t Be Updated to Python 3 in Time

  • With 35 million lines of Python code, the Athena trading platform is at the core of JPMorgan's business operations. A late start to migrating to Python 3 could create a security risk.
  • Athena platform is used internally at JPMorgan for pricing, trading, risk management, and analytics, with tools for data science and machine learning.
  • This extensive feature set utilizes over 150,000 Python modules, over 500 open source packages, and 35 million lines of Python code contributed by over 1,500 developers, according to data presented by Misha Tselman, executive director at J.P. Morgan Chase in a talk at PyData 2017.
  • And JPMorgan is going to miss the deadline
  • Roadmap puts "most strategic components" compatible with Python 3 by the end of Q1 2020
  • JPMorgan uses Continuous Delivery, with 10,000 to 15,000 production changes per week
  • "If you maintain a library that other developers depend on," the post states, "you may be preventing them from updating to 3. By holding other developers back, you are indirectly and likely unintentionally increasing the security risks of others," adding that developers who do not publish code publicly should "consider your colleagues who may also be using your code internally."

Brian #2: organize

  • suggested by Ariel Barkan
  • a Python based file management automation tool
  • configuration is via a yml file
  • command line tool to organize your file system
  • examples:
    • move all of your screenshots off of your desktop into a screenshots folder
    • move old incomplete downloads into trash
    • remove empty files from certain folders
    • organize receipts and invoices into date based folders

Michael #3: PEP 589 – TypedDict: Type Hints for Dictionaries With a Fixed Set of Keys

  • Author: Jukka Lehtosalo
  • Sponsor: Guido van Rossum
  • Status: Accepted
  • Version: 3.8
  • PEP 484 defines the type Dict[K, V] for uniform dictionaries, where each value has the same type, and arbitrary key values are supported.
  • It doesn't properly support the common pattern where the type of a dictionary value depends on the string value of the key.
  • Core idea: Consider creating a type to validate an arbitrary JSON document with a fixed schema
  • Proposed syntax:
from typing import TypedDict

class Movie(TypedDict):
name: str
year: int

movie: Movie = {'name': 'Blade Runner',
'year': 1982}
  • Operations on movie can be checked by a static type checker
movie['director'] = 'Ridley Scott' # Error: invalid key 'director'
movie['year'] = '1982' # Error: invalid value type ("int" expected)

Brian #4: gazpacho

  • gazpacho is a web scraping library
  • “It replaces requests and BeautifulSoup for most projects. “
  • “gazpacho is small, simple, fast, and consistent.”
  • example of using gazpacho to scrape hockey data for fantasy sports.
  • simple interface, short scripts, really beginner friendly
  • retrieve with get, parse with Soup.
  • I don’t think it will completely replace the other tools, but for simple get/parse/find operations, it may make for slimmer code.
  • Note, I needed to update certificates to get this to work. see this.

Michael #5: How pip install Works

  • via PyDist
  • What happens when you run pip install [somepackage]?
  • First pip needs to decide which distribution of the package to install.
    • This is more complex for Python than many other languages
  • There are 7 different kinds of distributions, but the most common these days are source distributions and binary wheels.
  • A binary wheel is a more complex archive format, which can contain compiled C extension code.
  • Compiling, say, numpy from source takes a long time (~4 minutes on my desktop), and it is hard for package authors to ensure that their source code will compile on other people's machines.
  • Most packages with C extensions will build multiple wheel distributions, and pip needs to decide which if any are suitable for your computer.
  • To find the distributions available, pip requests https://pypi.org/simple/[somepackage], which is a simple HTML page full of links, where the text of the link is the filename of the distribution.
  • To select a distribution, pip first determines which distributions are compatible with your system and implementation of python.
    • binary wheels, it parses the filenames according to PEP 425, extracting the python implementation, application binary interface, and platform.
    • All source distributions are assumed to be compatible, at least at this step in the process
  • Once pip has a list of compatible distributions, it sorts them by version, chooses the most recent version, and then chooses the "best" distribution for that version
  • It prefers binary wheels if there are any
  • Determining the dependencies for this distribution is not simple either.
  • For binary wheels, the dependencies are listed in a file called METADATA. But for source distributions the dependencies are effectively whatever gets installed when you execute their setup.py script with the install command.
  • What happens though if one of the distributions pip finds violates the requirements of another? It ignores the requirement and installs idna anyway!
  • Next pip has to actually build and install the package.
  • it needs to determine which library directory to install the package in—the system's, the user's, or a virtualenvs?
  • Controlled by sys.prefix, which in turn is controlled by pip's executable path and the PYTHONPATH and PYTHONHOME environment variables.
  • Finally, it moves the wheel files into the appropriate library directory, and compiles the python source files into bytecode for faster execution.
  • Now your package is installed!

Brian #6: daily pandas tricks

  • Kevin Markham is sending out one pandas tip or trick per day via twitter.
  • It’s been fun to watch and learn new bits.
  • The link is a sampling of a bunch of them.
  • Here’s just one example:
Need to rename all of your columns in the same way? Use a string method:

Replace spaces with _:
df.columns = df.columns.str.replace(' ', '_')

Make lowercase & remove trailing whitespace:
df.columns = df.columns.str.lower().str.rstrip()

Extras

Michael:

Jokes:

Oct 15 2019

26mins

Play

#151 Certified! It works on my machine

Podcast cover
Read more

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean

Brian #1: Python alternative to Docker

  • Matt Layman
  • Using Shiv, from LinkedIn
    • Mentioned briefly in episode 114
    • Shiv uses zipapp, PEP 441.
    • Execute code directly from a zip file.
    • App code and dependencies can be bundled together.
    • “Having one artifact eliminates the possibility of a bad interaction getting to your production system.”
    • article includes an example of all the steps for packaging a Django app with Gunicorn.
    • includes talking about deployment.
  • Matt includes shoutouts to:
    • Platform as a Service providers
    • Manual steps to do it all.
    • Docker
  • Compares the process against Docker and discusses when to choose one over the other.
  • Also an interesting read: Docker is in deep trouble

Michael #2: How to support open-source software and stay sane

  • via Jason Thomas written by Anna Nowogrodzki
  • Releasing lab-built open-source software often involves a mountain of unforeseen work for the developers.
  • Article opens: “On 10 April, astrophysicists announced that they had captured the first ever image of a black hole. This was exhilarating news, but none of the giddy headlines mentioned that the image would have been impossible without open-source software.”
  • The image was created using Matplotlib, a Python library for graphing data, as well as other components of the open-source Python ecosystem. Just five days later, the US National Science Foundation (NSF) rejected a grant proposal to support that ecosystem, saying that the software lacked sufficient impact.
  • Open-source software is widely acknowledged as crucially important in science, yet it is funded non-sustainably.
  • “It’s sort of the difference between having insurance and having a GoFundMe when their grandma goes to the hospital,” says Anne Carpenter
  • Challenges
    • Scientists writing open-source software often lack formal training in software engineering.
    • Yet poorly maintained software can waste time and effort, and hinder reproducibility.
  • If your research group is planning to release open-source software, you can prepare for the support work
  • Obsolescence isn’t bad, she adds: knowing when to stop supporting software is an important skill.
  • However long your software will be used for, good software-engineering practices and documentation are essential.
  • These include continuous integration systems (such as TravisCI), version control (Git) and unit testing.
  • To facilitate maintenance, Varoquaux recommends focusing on code readability over peak performance.

Brian #3: The Hippocratic License

  • Coraline Ada Ehmke
  • Interesting idea to derive from MIT, but add restrictions.
  • This license adds these restrictions:
    • “The software may not be used by individuals, corporations, governments, or other groups for systems or activities that actively and knowingly endanger, harm, or otherwise threaten the physical, mental, economic, or general well-being of individuals or groups in violation of the United Nations Universal Declaration of Human Rights”
  • I could see others with different restrictions, or this but more.

Michael #4: MATLAB vs Python: Why and How to Make the Switch

  • MATLAB® is widely known as a high-quality environment for any work that involves arrays, matrices, or linear algebra.
  • I personally used it for wavelet-decomposition of real time eye measurements during cognitively intensive human workloads… That toolbox costs $2000 per user.
  • Difference in philosophy: Closed, paid vs. open source.
  • Since Python is available at no cost, a much broader audience can use the code you develop
  • Also, there is GNU Octave is a free and open-source clone of MATLAB apparently

Brian #5: PyperCard - Easy GUIs for All

  • Nicholas Tollervey
  • Came up on episode 143
  • Also, episode 89 of Test & Code
  • Really easy to quickly set up a GUI specified by a list of “Card” objects. (different from cards project)
  • Simple examples are choose your own adventure type applications, where one button takes you to another card, and another button, a different card.
  • However, the “next card” could be a Python function that can do anything, as long as it returns a string with the name of the next card.
  • Lots of potential here, especially with input boxes, images, sound, and more.
  • Super fun, but also might have business use.

Michae #6: pynode

pynode.call('add', 1, 2, (err, result) => {
if (err) return console.log('error : ', err)
result === 3 // true
})

Jokes

The "Works on My Machine" Certification Program, get certified!

Oct 10 2019

25mins

Play

#150 Winning the Python software interview

Podcast cover
Read more

Sponsored by Datadog: pythonbytes.fm/datadog

Michael #1: How to Stand Out in a Python Coding Interview

  • Real Python, by James Timmins
  • Are tech interviews broken? Well at least we can try to succeed at them anyway
  • You’ve made it past the phone call with the recruiter, and now it’s time to show that you know how to solve problems with actual code…
  • Interviews aren’t just about solving problems: they’re also about showing that you can write clean production code. This means that you have a deep knowledge of Python’s built-in functionality and libraries.
  • Things to learn
    • Use enumerate() to iterate over both indices and values
    • Debug problematic code with breakpoint()
    • Format strings effectively with f-strings
    • Sort lists with custom arguments
    • Use generators instead of list comprehensions to conserve memory
    • Define default values when looking up dictionary keys
    • Count hashable objects with the collections.Counter class
    • Use the standard library to get lists of permutations and combinations

Brian #2: The Python Software Foundation has updated its Code of Conduct

  • There’s now one code of conduct for PSF and PyCon US and other spaces sponsored by the PSF
  • This includes some regional conferences, such as PyCascades, and some meetup groups, (ears perk up)
  • The docs
  • Do we need to care?
    • all of us, yes. If there weren’t problems, we wouldn’t need these.
    • attendees, yes. Know before you go.
    • organizers, yes. Better to think about it ahead of time and have a plan than have to make up a strategy during an event if something happens.
    • me, in particular, and Michael. Ugh. yes. our first meetup is next month. I’d like to be in line with the rest of Python. So, yep, we are going to have to talk about this and put something in place.

Michael #3: The Interview Study Guide For Software Engineers

Brian #4: re-assert : “show where your regex match assertion failed”

  • Anthony Sotille
  • re-assert provides a helper class to make assertions of regexes simpler.”
  • The Matches objects allows for useful pytest assertion messages
  • In order to get my head around it, I looked at the test code:
def test_match_old():
> assert re.match('foo', 'fob')
E AssertionError: assert None
E + where None = [HTML_REMOVED]('foo', 'fob')
E + where [HTML_REMOVED] = re.match

test_re.py:8: AssertionError
____________ test_match_new ___________________

def test_match_new():
> assert Matches('foo') == 'fob'
E AssertionError: assert Matches('foo') ^ == 'fob'
E -Matches('foo')
E - # regex failed to match at:
E - #
E - #> fob
E - # ^
E +'fob'

Michael #5: awesome-python-typing

Brian #6: Developer Advocacy: Frequently Asked Questions

  • Dustin Ingram
  • I know a handful of people who have this job title. What is it?
  • disclaimer: Dustin is a DA at Google. Other companies might be different
  • What is it?
    • “I help represent the Python community at [company]"
    • “part of my job is to be deeply involved in the Python community.”
    • working on projects that help Python, PyPI, packaging, etc.
    • speaking at conferences
    • talking to people. customers and non-customers
    • talking to product teams
    • being “user zero” for new products and features
    • paying attention to places users might raise issues about products
    • working in open source
    • creating content for Python devs
    • being involved in the community as a company rep
    • representing Python in the company
    • coordinating with other DAs
  • Work/life?

    • Not all DAs travel all the time. that was my main question.
  • Talk Python episode: War Stories of the Developer Evangelists

Extras:

Michael:

Joke:

via https://twitter.com/NotGbo/status/1173667028965777410

Web Dev Merit Badges

Oct 05 2019

23mins

Play

#149 Python's small object allocator and other memory features

Podcast cover
Read more

Sponsored by Datadog: pythonbytes.fm/datadog

Brian #1: Dropbox: Our journey to type checking 4 million lines of Python

  • Continuing saga, but this is a cool write up.
  • Benefits
    • “Experience tells us that understanding code becomes the key to maintaining developer productivity. Without type annotations, basic reasoning such as figuring out the valid arguments to a function, or the possible return value types, becomes a hard problem. Here are typical questions that are often tricky to answer without type annotations:
      • Can this function return None?
      • What is this items argument supposed to be?
      • What is the type of the id attribute: is it int, str, or perhaps some custom type?
      • Does this argument need to be a list, or can I give a tuple or a set?”
    • Type checker will find many subtle bugs.
    • Refactoring is easier.
    • Running type checking is faster than running large suites of unit tests, so feedback can be faster.
    • Typing helps IDEs with better completion, static error checking, and more.
  • Long story, but really cool learnings of how and why to tackle adding type hints to a large project with many developers.
  • Conclusion. mypy is great now, because DropBox needed it to be.

Michael #2: Setting Up a Flask Application in Visual Studio Code

  • Video, but also as a post
  • Follow on to the same in PyCharm:
  • Steps outside VS Code
    • Clone repo
    • Create a virtual env (via venv)
    • Install requirements (via requirements.txt)
    • Setup flask app ENV variable
    • flask deploy ← custom command for DB
  • VS Code
    • Open the folder where the repo and venv live
    • Open any Python file to trigger the Python subsystem
    • Ensure the correct VENV is selected (bottom left)
    • Open the debugger tab, add config, pick Flask, choose your app.py file
    • Debug menu, start without debugging (or with)
  • Adding tests via VS Code
    • Open command pallet (CMD SHIFT P), Python: Discover Tests, select framework, select directory of tests, file pattern, new tests bottle on the right bar

Brian #3: Multiprocessing vs. Threading in Python: What Every Data Scientist Needs to Know

  • How data scientists can go about choosing between the multiprocessing and threading and which factors should be kept in mind while doing so.
  • Does not consider async, but still some great info.
  • Overview of both concepts in general and some of the pitfalls of parallel computing.
  • The specifics in Python, with the GIL
  • Use threads for waiting on IO or waiting on users.
  • Use multiprocessing for CPU intensive work.
  • The surprising bit for me was the benchmarks
    • Using something speeds up the code. That’s obvious.
    • The difference between the two isn’t as great as I would have expected.
  • A discussion of merits and benefits of both.
  • And from the perspective of data science.
  • A few more examples, with code, included.

Michael #4: ORM - async ORM

  • And https://github.com/encode/databases
  • The orm package is an async ORM for Python, with support for Postgres, MySQL, and SQLite.
  • SQLAlchemy core for query building.
  • databases for cross-database async support.
  • typesystem for data validation.
  • Because ORM is built on SQLAlchemy core, you can use Alembic to provide database migrations.
  • Need to be pretty async savy

Brian #5: Getting Started with APIs

  • dataquest.io post
  • Conceptual introduction of web APIs
  • Discussion of GET status codes, including a nice list with descriptions.
    • examples:
      • 301: The server is redirecting you to a different endpoint. This can happen when a company switches domain names, or an endpoint name is changed.
      • 400: The server thinks you made a bad request. This can happen when you don’t send along the right data, among other things.
  • endpoints
  • endpoints that take query parameters
  • JSON data
  • Examples in Python for using:
    • requests to query endpoints.
    • json to load and dump JSON data.

Michael #6: Memory management in Python

  • This article describes memory management in Python 3.6.
  • Everything in Python is an object. Some objects can hold other objects, such as lists, tuples, dicts, classes, etc.
  • such an approach requires a lot of small memory allocations
  • To speed-up memory operations and reduce fragmentation Python uses a special manager on top of the general-purpose allocator, called PyMalloc.
  • Layered managers
    • RAM
    • OS VMM
    • C-malloc
    • PyMem
    • Python Object allocator
    • Object memory
  • Three levels of organization
    • To reduce overhead for small objects (less than 512 bytes) Python sub-allocates big blocks of memory.
    • Larger objects are routed to standard C allocator.
    • three levels of abstraction — arena, pool, and block.
    • Block is a chunk of memory of a certain size. Each block can keep only one Python object of a fixed size. The size of the block can vary from 8 to 512 bytes and must be a multiple of eight
    • A collection of blocks of the same size is called a pool. Normally, the size of the pool is equal to the size of a memory page, i.e., 4Kb.
    • The arena is a chunk of 256kB memory allocated on the heap, which provides memory for 64 pools.
  • Python's small object manager rarely returns memory back to the Operating System.
  • An arena gets fully released If and only if all the pools in it are empty.

Extras

Brian:

  • Tuesday, Oct 6, Python PDX West,
  • Thursday, Sept 26, I’ll be speaking at PDX Python, downtown.
  • Both events, mostly, I’ll be working on new programming jokes unless I come up with something better. :)

Michael:

Jokes:
A few I liked from the dad joke list.

  • What do you call a 3.14 foot long snake? A π-thon
  • What if it’s 3.14 inches, instead of feet? A μ-π-thon
  • Why doesn't Hollywood make more Big Data movies? NoSQL.
  • Why didn't the div get invited to the dinner party? Because it had no class.

Sep 25 2019

37mins

Play

#148 The ASGI revolution is upon us!

Podcast cover
Read more

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean

Brian #1: Annual Release Cycle for Python - PEP 602

  • Under discussion
  • Annual release cadence
  • Seventeen months to develop a major version
    • 5 months unversioned, 7 months alpha releases when new features and fixes come in, 4 months of betas with no new features, 1 month final RCs.
  • One year of full support, four more years of security fixes.
  • Rationale/Goals
    • smaller releases
    • features and fixes to users sooner
    • more gradual upgrade path
    • predictable calendar releases that line up will with sprints and PyConUS
    • explicit alpha phase
    • decrease pressure and rush to get features into beta 1
  • Risks
    • Increase concurrent supported versions from 4 to 5.
    • Test matrix increase for integrators and distributions.
  • PEP includes rejected ideas like:
    • 9 month cadence
    • keep 18 month cadence

Michael #2: awesome-asgi

  • A curated list of awesome ASGI servers, frameworks, apps, libraries, and other resources
  • ASGI is a standard interface positioned as a spiritual successor to WSGI. It enables communication and interoperability across the whole Python async web stack: servers, applications, middleware, and individual components.
  • Born in 2016 to power the Django Channels project, ASGI and its ecosystem have been expanding ever since, boosted by the arrival of projects such as Starlette and Uvicorn in 2018.
  • Frameworks for building ASGI web applications.
    • Bocadillo - Fast, scalable and real-time capable web APIs for everyone. Powered by Starlette. Supports HTTP (incl. SSE) and WebSockets.
    • Channels - Asynchronous support for Django, and the original driving force behind the ASGI project. Supports HTTP and WebSockets with Django integration, and any protocol with ASGI-native code.
    • FastAPI - A modern, high-performance web framework for building APIs with Python 3.6+ based on standard Python type hints. Powered by Starlette and Pydantic. Supports HTTP and WebSockets.
    • Quart - A Python ASGI web microframework whose API is a superset of the Flask API. Supports HTTP (incl. SSE and HTTP/2 server push) and WebSockets.
    • Responder - A familiar HTTP Service Framework for Python, powered by Starlette. (ASGI 2.0 only, ed.)
    • Starlette - Starlette is a lightweight ASGI framework/toolkit, which is ideal for building high performance asyncio services. Supports HTTP and WebSockets.

Brian #3: Jupyter meets the Earth

  • Lindsey Heagy & Fernando Pérez
  • “We are thrilled to announce that the NSF is funding our EarthCube proposal “Jupyter meets the Earth: Enabling discovery in geoscience through interactive computing at scale”
  • “This project provides our team with $2 Million in funding over 3 years as a part of the NSF EarthCube program. It also represents the first time federal funding is being allocated for the development of core Jupyter infrastructure.”
  • “Our project team includes members from the Jupyter and Pangeo communities, with representation across the geosciences including climate modeling, water resource applications, and geophysics. Three active research projects, one in each domain, will motivate developments in the Jupyter and Pangeo ecosystems. Each of these research applications demonstrates aspects of a research workflow which requires scalable, interactive computational tools.”
  • “The adoption of open languages such as Python and the coalescence of communities of practice around open-source tools, is visible in nearly every domain of science. This is a fundamental shift in how science is conducted and shared.”
  • Geoscience use cases
    • climate data analysis
    • hydrologic modeling
    • geophysical inversions
  • User-Centered Development
    • data discovery
    • scientific discovery through interactive computing
    • established tools and data visualization
    • using and managing shared computational infrastructure

Michael #4: Asynchronous Django

  • via Jose Nario
  • Python compatibility
    • Django 3.0 supports Python 3.6, 3.7, and 3.8. We highly recommend and only officially support the latest release of each series
    • The Django 2.2.x series is the last to support Python 3.5.
  • Other items but
  • Big news: ASGI support
  • Django 3.0 begins our journey to making Django fully async-capable by providing support for running as an ASGI application.
  • This is in addition to our existing WSGI support. Django intends to support both for the foreseeable future.
  • Note that as a side-effect of this change, Django is now aware of asynchronous event loops and will block you calling code marked as “async unsafe” - such as ORM operations - from an asynchronous context.

Brian #5: The 1x Engineer

  • Fun take on 10x. List actually looks like probably a 3-4x to me, maybe even 8x or more. How high does this scale go anyway?
  • non-exhaustive list qualities, here’s a few.
    • Has a life outside engineering.
    • Writes code that others can read.
    • Doesn't act surprised when someone doesn’t know something.
    • Asks for help when they need it.
    • Is able to say "I don't know."
    • Asks questions.
    • Constructively participates in code reviews.
    • Can collaborate with others.
    • Supports code, even if they did not write it.
    • Can feel like an impostor at times.
    • Shares knowledge.
    • Never stops learning.
      • [obviously listens to Python Bytes, Talk Python, and Test & Code]
    • Is willing to leave their comfort zone.
    • Contributes to the community.
    • Has productive and unproductive days.
    • Doesn't take themselves too seriously.
    • Fails from time to time.
    • Has a favorite editor, browser, and operating system, but realizes others do too.

Michael #6: Sunsetting Python 2

  • January 1, 2020, will be the day that we sunset Python 2
  • Why are you doing this? We need to sunset Python 2 so we can help Python users.
  • How long is it till the sunset date? pythonclock.org will tell you.
  • What will happen if I do not upgrade by January 1st, 2020? If people find catastrophic security problems in Python 2, or in software written in Python 2, then most volunteers will not help fix them.
  • I wrote code in Python 2. How should I port it to Python 3? Please read the official "Porting Python 2 Code to Python 3" guide.
  • I didn't hear anything about this till just now. Where did you announce it? We talked about it at software conferences, on the Python announcement mailing list, on official Python blogs, in textbooks and technical articles, on social media, and to companies that sell Python support.

Extras

Brian:

  • working on a Portland Westside Python Meetup, info will be at pythonpdx.com
    • Hoping to get something ready for Oct. But… if not, hopefully by Nov.

Michael:

Sep 18 2019

24mins

Play

#147 Mocking out AWS APIs

Podcast cover
Read more

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean

Brian #1: rapidtables

  • rapidtables … converts lists of dictionaries to pre-formatted tables. And it does the job as fast as possible.”
  • Also can do color formatting if used in conjunction with termcolor.colored, but I’m mostly excited about really easily generating tabular data with print.
  • Can also format to markdown or reStructured text, and can do alignment, …

Michael #2: httpx

Brian #3: Quick and dirty mock service with Starlette

  • Matt Layman
  • Mock out / fake a third party service in a testing environment.
  • Starlette looks fun, but the process can be used with other API producing server packages.
  • We tell people to do things like this all the time, but there are few examples showing how to.
  • This example also introduces a delay because the service used in production takes over a minute and part of the testing is to make sure the system under test handles that delay gracefully.
  • Very cool, easy to follow write up. (Should probably have Matt on a Test & Code episode to talk about this strategy.)

Michael #4: Mocking out AWS APIs

  • via Giuseppe Cunsolo
  • A library that allows you to easily mock out tests based on AWS infrastructure.
  • Lovely use of a decorator to mock out S3
  • Moto isn't just for Python code and it isn't just for S3. Look at the standalone server mode for more information about running Moto with other languages.
  • Be sure to check out very important note.

Brian #5: μMongo: sync/async ODM

  • “μMongo is a Python MongoDB ODM. It inception comes from two needs: the lack of async ODM and the difficulty to do document (un)serialization with existing ODMs.”
  • works with common mongo drivers such as PyMongo, TxMongo, motor_asyncio, and mongomock. (Hadn’t heard of mongomock before, I’ll have to try that out.)
  • Note: We’ve discussed MongoEngine before. (I’m curious what Michael has to say about uMongo.)

Michael #6: Single Responsibility Principle in Python

Extras

Michael:

Jokes

  • Q: What do you get when you cross a computer and a life guard?
  • A: A screensaver!

  • Q: What do you get when you cross a computer with an elephant?

  • A: Lots of memory!

via https://github.com/wesbos/dad-jokes

Anti-joke (we ready for those yet?): A Python developer, a PHP developer, a C# developer, and a Go developer went to lunch together. They had a nice lunch and got along fine.

Sep 11 2019

25mins

Play

#146 Slay the dragon, learn the Python

Podcast cover
Read more

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean

Special guest: Trey Hunner

Brian #1: Positional-only arguments in Python

  • by Sanket
  • Feature in 3.8
  • “To specify arguments as positional-only, a / marker should be added after all those arguments in the function definition. “
  • Great example of a pow(x, y, /, mod=None) function where the names x and y are meaningless and if given in the wrong order would be confusing.

Trey #2: django-stubs

  • Type checking for Django!
  • It’s new and from my very quick testing on my Django site it definitely doesn’t work with everything yet, but it’s promising
  • I don’t use type annotations in Django yet, but I very well might eventually

Michael #3: CodeCombat

  • Super fun game for learning to code
  • Real code but incredibly easy coding
  • Can subscribe or use the free tier
  • Just got $6M VC funding

Brian #4: Four Use Cases for When to Use Celery in a Flask Application

  • or really any web framework
  • by Nick Janetakis
  • “Celery helps you run code asynchronously or on a periodic schedule which are very common things you'd want to do in most web projects.”
  • examples:
    • sending emails out
    • connecting to 3rd party APIs.
    • performing long running tasks. Like, really long.
    • Running tasks on a schedule.

Trey #5: pytest-steps

  • Created by smarie
  • Can use a generator syntax with yield statements to break a big test up into multiple named “steps” that’ll show up in your pytest output
  • If one step fails, the rest of the steps will be skipped by default
  • You can also customize it to make optional steps, which aren’t required for future steps to run, or steps which depend on other optional steps explicitly
  • The documentation shows a lot of different ways to use it, but the generator approach looks by far the most readable to me (also state is shared between steps with this approach whereas the others require some fancy state-capturing object which looks confusing to me)
  • I haven’t tried this, but my use case would be my end-to-end/functional tests, which would work great with steps because I’m often using Selenium to navigate between a number of different pages and forms, one click at a time.

Michael #6: docassemble

  • Created by Jonathan Pyle
  • A free, open-source expert system for guided interviews and document assembly, based on Python, YAML, and Markdown.
  • Features
    1. WYSIWYG: Compose your templates in .docx (with help of a Word Add-in) or .pdf files.
    2. Signatures: Gather touchscreen signatures and embed them in documents.
    3. Live chat: Assist users in real time with live chat, screen sharing, and remote screen control.
    4. AI: Use machine learning to process user input.
    5. SMS: Send text messages to your users
    6. E-mail: Send and receive e-mails in your interviews.
    7. OCR: Use optical character recognition to process images uploaded by the user.
    8. Multilingual: Offer interviews in multiple languages.
    9. Multiuser: Develop applications that involve more than one user, such as mediation or counseling interviews.
    10. Extensible: Use the power of Python to extend the capabilities of your interviews.
    11. Open: Package your interviews and use GitHub and PyPI to share your work with the docassemble user community.
    12. Background Tasks: Do things behind the scenes of the interview, even when the user is not logged in.
    13. Scalable: Deploy your interviews on multiple machines to handle high traffic.
    14. Secure: Protect user information with server-side encryption, two-factor authentication, document redaction, and other security features.
    15. APIs: Integrate with third-party applications using the API, or send your interviews input and extract output using code.

Extras

Michael:

Joke

via Avram Lubkin

Knock! Knock!
Who's there?
Recursive function.
Recursive function who?
Knock! Knock!

Nice. to get that joke, you’ll have to understand recursion.
to understand recursion,

  • either google “recursion”, and click on “did you mean “recursion””
  • learn it in small steps. step one, recursion

text conversation:

  • first person: “Hey, what’s your address?”
  • second: gives IP address
  • first: “No. Your local address”
  • second: 127.0.0.1
  • first: “No. Your physical address”
  • second: gives MAC address

Sep 08 2019

23mins

Play

#145 The Python 3 “Y2K” problem

Podcast cover
Read more

Sponsored by Datadog: pythonbytes.fm/datadog

Special guests

Michael #1: friendly-traceback

  • via Jose Carlos Garcia (I think 🙂 )
  • Aimed at Python beginners: replacing standard traceback by something easier to understand
  • Shows help for exception type
  • Shows local variable values
  • Shows code in a cleaner form with more context
  • 3 ways to install
    • As an exception hook
    • Explicit explain
    • When running an app

Matt #2: Pandas Users Survey

  • Most use it almost everyday but have less than 2 years experience
  • Linux 61%, Windows 60%, Mac 42%
  • 93% Python 3

Anthony #3: python3 “Y2K” problem (python3.10 / python4.0)

  • with python3.8 close to release and python3.9 right around the corner, what comes after?
  • both python3.10 and python4.0 present some problems
    • sys.version[:3] which will suddenly report '``3.1``' in 3.10
    • a lot of code (including six.PY3!) uses sys.version_info[0] == 3 which will suddenly be false in python4.0 (and start running python2 code!)
  • early-to-mid 2020 we should start seeing the next version in the wild as python3.9 reaches beta
  • easy ways to start testing this early:
    • python3.10 - a build of cpython for ubuntu with the version number changed
    • flake8-2020 - a flake8 plugin which checks for these common issues-

Michael #4: pypi research

  • via Adam (Codependent Codr)
  • Really interesting research paper on the current state of Pypi from a couple authors at the University of Michigan: "An Empirical Analysis of the Python Package Index" - https://arxiv.org/pdf/1907.11073.pdf
  • Comprehensive empirical summary of the Python Package Repository, PyPI, including both package metadata and source code covering 178,592 packages, 1,745,744 releases, 76,997 contributors, and 156,816,750 import statements.
  • We provide counts and trends for packages, releases, dependencies, category classifications, licenses, and package imports, as well as authors, maintainers, and organizations.
  • Within PyPI, we find that the growth of the repository has been robust under all measures, with a compound annual growth rate of 47% for active packages, 39% for new authors, and 61% for new import statements over the last 15 years.
  • In 2005, there were 96 active packages, 96!
  • MIT is the most common license
  • (Matt) I saw this and was surprised at most commonly used libraries. What do you think the most common 3rd party library is?

Matt #5: DaPy

  • “Pandas for humans” - Matt’s words
  • Has portions of pandas, scikit-learn, yellowbrick, and numpy
  • Designed for “data analysis, not for coders”

Anthony #6: python-remote-pdb

  • very small over-the-network remote debugger
  • thin wrapper around pdb in a single file (easy to drop the file on PYTHONPATH if you can’t pip install)
  • not as fully featured as other remote debuggers such as pudb / rpdb / pycharm’s debugger but very easy to drop in
  • fully supports [breakpoint()](https://www.python.org/dev/peps/pep-0553/) (python3.7+ or via future-breakpoint)
  • access pdb via telnet / nc / socat
  • I’m using it to debug a text editor I’m writing to learn curses!

Extras:

Michael:

Matt

Anthony:

Jokes:

Michael: Two mathematicians are sitting at a table in a pub having an argument about the level of math education among the general public.

The one defending overall math knowledge gets up to go to the washroom. On the way back, he encounters their waitress and says, "I'll add an extra $10 to your tip, if you'll answer a question for me when I ask it. All you have to say is 'x-squared'." She agrees.

A few minutes later the populist mathematician says to his buddy, "I'll bet you $20 that even our waitress can tell us the integral of 2x."
The cynic agrees to the bet.

So the schemer beckons the waitress to their table and asks the question, to which she replies "x-squared". As he begins to gloat and demand his winnings, the waitress continues, "Plus a constant."

Anthony: I had a golang joke prepared, but then I panic()d

Aug 31 2019

34mins

Play

#144 Are you mocking me? It won't work!

Podcast cover
Read more

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean

Chris #1: Why your mock doesn’t work

  • Ned Batchelder
  • TDD is an important practice for development, and as my team is finding out, mocking objects is not as easy at it seems at first.
  • I love that Ned gives an overview of how Mock works
  • But also gives two resources to show you alternatives to Mock, when you really don’t need it.
  • From reading these articles and video, I’ve learned that it’s hard to make mocks but it’s important to:
    • Create only one mock for each object you’re mocking
    • that mocks only what you need
    • have tests that run the mock against your code and your mock against the third party

Mahmoud #2: Vermin

  • By Morten Kristensen
  • Rules-based Python version compatibility detector
  • caniuse is cool, but it’s based on classifiers. When it comes to your own code, it’ll only tell you what you tell it.
  • If you’ve got legacy libraries, or like most companies, an application, then you’ll need something more powerful.
  • Vermin tells you the minimum compatible Python version, all the way down to the module and even function level.

Brian #3: The nonlocal statement in Python

  • Abhilash Raj
  • When global is too big of a hammer.
  • This doesn’t work:
def function():
x = 100
def incr(y):
x = x + y
incr(100)
  • This does:
def function():
x = 100
def incr(y):
nonlocal x
x = x + y
incr(100)
print(x)

Chris #4: twitter.com/brettsky/status/1163860672762933249

  • Brett Cannon
  • Microsoft Azure improves python support
    • 2 key points about the new Python support in Azure Functions:
      • it's debuting w/ 3.6, but 3.7 support is actively being worked on and 3.8 support won't take nearly as long, and
      • native async/await support!

Mahmoud #5: Awesome Python Applications update

Brian #6: pre-commit now has a quick start guide

  • Wanna use pre-commit but don’t know how to start? Here ya go!
  • Runs through
    • install
    • configuration
    • installing hooks
    • running hooks against your project
  • I’d like to add
    • Add hooks to your project one at a time
    • For each new hook
      • add to pre-commit-config.yml
      • run pre-commit install to install hook
      • run pre-commit run --``all-files
      • review changes made to your project
        • if good, commit
        • if bad
        • revert
        • modify config of tools, such as pyproject.toml for black, .flake8 for flake8, etc.
        • try again

Extras

Chris:

Mahmoud:

  • PyGotham 2019 October (Maintainers Conf in Washington DC, too)
  • Real Python Pandas course

Brian:

Jokes

  • I was looking for some programming one liners online; looked on a reddit thread; read a great answer; which was “any joke can be a one-liner with enough semicolons.”
  • A SQL statement walks into to a bar and up to two tables and asks, “Mind if I join you?”

Aug 23 2019

25mins

Play

#143 Spike the robot, powered by Python!

Podcast cover
Read more

Special guest: Kelly Schuster-Paredes

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean

Brian #1: Keynote: Python 2020 - Łukasz Langa - PyLondinium19

  • Enabling Python on new platforms is important.
  • Python needs to expand further than just CPython.
    • Web, 3D games, system orchestration, mobile, all have other languages that are more used. Perhaps it’s because the full Python language, like CPython in full is more than is needed, and a limited language is necessary.
  • MicroPython and CircuitPython are successful.
    • They are limited implementations of Python
  • Łukasz talks about many parts of Python that could probably be trimmed to make targeted platforms very usable without losing too much.
  • It’d be great if more projects tried to implement Python versions for other platforms, even if the Python implementation is limited.

Kelly #2: Mu Editor

  • by Nicholas Tollervey
  • Lots of updates happening to the Code with Mu software
  • Mu is a Python code editor for beginner programmers
  • Code with Mu presented at EuroPython and shared a lot of interesting updates and things in the alpha version of Mu, available on code with Mu website.
  • Mu is a modal editor:
    • BBC Microbit
    • Circuit Python
    • ESP Micropython
    • Pygame Zero
    • Python 3
      • Tiago Monte’s recorded presentation at EuroPython
      • Game with Turtle
    • Flask — release notes
  • Made with Mu at EuroPython videos
  • Hot off the press: Nick just released Pypercard a HyperCard inspired GUI framework for BEGINNER developers in Python based off of Adafruit’s release.
    • It is a “PyperCard is a HyperCard inspired Pythonic and deliberately constrained GUI framework for beginner programmers.
    • linked repos on GitHub.
    • module re-uses the JSON specification used to create HyperCard
    • The concept allows user to “create Hypercard like stacks of states” to allow beginner coders to create choose their own adventure games.

Michael #3: Understanding the Python Traceback

  • by Chad Hansen
  • The Python traceback has a wealth of information that can help you diagnose and fix the reason for the exception being raised in your code.
  • What do we learn right away?
    • The type of error
    • A description of the error (hopefully, sometimes)
    • The line of code the error occurred on
    • The call stack (filenames, line numbers, and module names)
    • If the error happened while handling another error
  • Read from bottom to top — that was weird to me
  • Most common error? AttributeError: 'NoneType' object has no attribute 'an_attribute'
  • Article talks about other common errors
  • Are you creating custom exceptions to make your packages more useful?

Brian #4: My oh my, flake8-mypy and pytest-mypy

  • contributed by Ray Cote via email
  • “For some reason, I continually have problems running mypy, getting it to look at the correct paths, etc. However, when I run it from flake8-mypy, I'm getting reasonable, actionable output that is helping me slowly type hint my code (and shake out a few bugs in the process). There's also a pytest-mypy, which I've not yet tried. “ - Ray
  • flake8-mypy **
    • Maintained by Łukasz Langa
    • “The idea is to enable limited type checking as a linter inside editors and other tools that already support Flake8 warning syntax and config.”
  • pytest-mypy
    • Maintained by Dan Bader and David Tucker
    • “Runs the mypy static type checker on your source files as part of your pytest test runs.”
      • Remind me to do a PR against the README to make pytest lowercase.

Kelly #5: Lego Education and Spike

  • In March of this year, Lego Education gave news of a new robot being released since the EV3 released of Mindstorms in 2013.
    • Currently the EV3 Mindstorm can be coded with Python and it is assumed that Spike Prime can be as well.
  • The current EV3 robots can currently be coded in python thanks to Nigel Ward. He created a site back in 2016 or earlier; through a program called the EV3Dev project.
  • Until recently, Lego had not endorsed the use of Python or had they released documentation.
    • Lego released a Getting started with EV3 MicroPython 59 page guide Version 1.0.0
    • EV3 MicroPython runs on top of ev3dev with a new Pybricks MicroPython runtime and library.
    • has its own Visual Studio Code extension
    • no need for terminal
    • Has instruction and lists of different features and classes used to program the PyBricks API- A python wrapper for the Databricks Rest API.
  • This opens up opportunities for students that compete in the First Lego League Competition to code in Python.
  • Example code for the Gyrobot

Michael #6: Python 3 at Mozilla

  • From January 2019.
  • Mozilla uses a lot of Python.
  • In mozilla-central there are over 3500 Python files (excluding third party files), comprising roughly 230k lines of code.
  • Additionally there are 462 repositories labelled with Python in the Mozilla org on Github
  • That’s a lot of Python, and most of it is Python 2.
  • But before tackling those questions, I want to address another one that often comes up right off the bat: Do we need to be 100% migrated by Python 2’s EOL?
  • No. But punting the migration into the indefinite future would be a big mistake:
    • Python 2 will no longer receive security fixes.
    • All of the third party packages we rely on (and there are a lot of them) will also stop being supported
    • Delaying means more code to migrate
    • Opportunity cost: Python 3 was first released in 2008 and in that time there have been a huge number of features and improvements that are not available in Python 2.
  • The best time to get serious about migrating to Python 3 was five years ago. The second best time is now.
  • Moving to Python 3
  • We stood up some linters.
    • One linter that makes sure Python files can at least get imported in Python 3 without failing
    • One that makes sure Python 2 files use appropriate __future__ statements to make migrating that file slightly easier in the future.
  • Pipenv & poetry & Jetty: a little experiment I’ve been building. It is a very thin wrapper around Poetry

Extras

Brian:

  • Python 3.8.0b3
    • “We strongly encourage maintainers of third-party Python projects to test with 3.8 during the beta phase and report issues …”

Michael:

Kelly:

Jokes

  • via Real Python and Nick Spirit
  • Python private method → Joke cartoon image.

Aug 14 2019

33mins

Play

#142 There's a bandit in the Python space

Podcast cover
Read more

Special guest: Brett Thomas

Sponsored by Datadog: pythonbytes.fm/datadog

Brian #1: Writing sustainable Python scripts

  • Vincent Bernat
  • Turning a quick Python script into a maintainable bit of software.
  • Topics covered:
    • Documentation as a docstring helps future users/maintainers know what problem you are solving.
    • CLI arguments with defaults instead of hardcoded values help extend the usability of the script.
    • Logging. Including debug logging (and how to turn them on with CLI arguments), and system logging for unattended scripts.
    • Tests. Simple doctests, and pytest tests utilizing parametrize to have one test and many test cases.

Brett #2: Static Analysis and Bandit

Michael #3: jupyter-black

  • Black formatter for Jupyter Notebook
  • One of the big gripes I have about these online editors is their formatting (often entirely absent)
  • Then the extension provides
    • a toolbar button
    • a keyboard shortcut for reformatting the current code-cell (default: Ctrl-B)
    • a keyboard shortcut for reformatting whole code-cells (default: Ctrl-Shift-B)

Brian #4: Report Generation workflow with papermill, jupyter, rclone, nbconvert, …

Brett #5: Rant on time deltas

Michael #6: How — and why — you should use Python Generators

  • by Radu Raicea
  • Generator functions allow you to declare a function that behaves like an iterator.
  • They allow programmers to make an iterator in a fast, easy, and clean way.
  • They only compute it when you ask for it. This is known as lazy evaluation.
  • If you’re not using generators, you’re missing a powerful feature
  • Often they result in simpler code than with lists and standard functions

Extras

Brian:

Michael:

Jokes

A good programmer is someone who always looks both ways before crossing a one-way street.

(reminds me of another joke: Adulthood is like looking both ways before crossing the street, then getting hit by an airplane)

Little bobby tables

Aug 06 2019

30mins

Play

#141 Debugging with f-strings coming in Python 3.8

Podcast cover
Read more

Sponsored by Datadog: pythonbytes.fm/datadog

Brian #1: Debugging with f-strings in Python 3.8

  • We’ve talked about the walrus operator, :=, but not yet “debug support for f-strings”
  • this: print(f'foo={foo} bar={bar}')
  • can change to this: print(f'{foo=} {bar=}')
  • and if you don’t want to print with repr() you can have str() be used with !s.
    • print(f'{foo=!s} {bar=!s}')
  • also !f can be used for float modifiers:
>>> import math
>>> print(f'{math.pi=!f:.2f}')
math.pi=3.14
  • one more feature, space preservation in the f-string expressions:
>>> a = 37
>>> print(f'{a = }, {a = }')
a = 37, a = 37

Michael #2: Am I "real" software developer yet?

  • by Sun-Li Beatteay
  • To new programmers joining the field, especially those without CS degrees, it can feel like the title is safe-guarded. Only bestowed on the select that have proven themselves.
  • Sometimes manifests itself as Impostor Syndrome
  • Focused on front-end development as I had heard that HTML, CSS and JavaScript were easy to pick up
  • That was when I decided to create a portfolio site for my wife, who was a product designer.
  • Did my best to surround myself with tech culture.
    • Watched YouTube videos
    • listened to podcasts
    • read blog posts from experienced engineers to keep myself motivated.
    • Daydreamed what it would be like to stand in their shoes.
  • My wife’s website went live in July of that year. I had done it.
  • Could I finally start calling myself something of a Software Engineer?
    • “Web development isn’t real programming”
  • Spent the next 18 months studying software development full time. I quit my job and moved in with my in-laws — which was a journey in-and-of itself.
    • Software engineer after 1-2 years? No so fast (says the internet)
  • The solution that I found for myself was simple yet terrifying: talking to people
  • MK: BTW, I don’t really like the term “engineer”

Brian #3: Debugging with local variables and snoop

  • debugging tools
  • ex: “You want to know which lines are running and which aren't, and what the values of the local variables are.”
    • Throw a @snoop decorator on a function and the function lines and local variable values will be dumped to stderr during run. Even showing loops a bunch of times.
  • It’s tools to almost debug as if you had a debugger, without a debugger, and without having to add a bunch of logging or print statements.
  • Lots of other use models to allow more focus.
    • wrap just part of your function with a with snoop block
    • only watch certain local variables.
    • turn off reporting for deep function/block levels.

Michael #4: New home for Humans

  • This came out of the blue with some trepidation:
  • kennethreitz commented 6 days ago:

In the spirit of transparency, I'd like to (publicly) find a new home for my repositories. I want to be able to still make contributions to them, but no longer be considered the "owner" or "arbiter" or "BDFL" of these repositories.

Some notable repos:

Brian #5: The Backwards Commercial License

  • Eran Hammer - open source dev, including hapi.js
  • Interesting idea to make open source projects maintainable
  • Three phases of software lifecycle for some projects:
    • first: project created to fill a need in one project/team/company, a single use case
    • second: used by many, active community, growing audience
    • three: work feels finished. bug fixes, security issues, minor features continue, but most people can stay on old stable versions
  • During the “done” phase, companies would like to have bug fixes but don’t want to have to keep changing their code to keep up.
  • Idea: commercial license to support old stable versions.
    • “If you keep up with the latest version, you do not require a license (unless you want the additional benefits it will provide).”
    • “However, very few companies can quickly migrate every time there is a new major release of a core component. Engineering resources are limited and in most cases, are better directed at building great products than upgrading supporting infrastructure. The backwards license provides this exact assurance. You can stay on any version you would like knowing that you are still running supported, well-maintained, and secure code.”
    • “The new commercial license will include additional benefits focused on providing enterprise customers the assurances needed to rely on these critical components for many years to come. “

Michael #6: Switching Python Parsers?

  • via Gi Bi, article by Guido van Rossum
  • Alternative to the home-grown parser generator that I developed 30 years ago when I started working on Python. (That parser generator, dubbed “pgen”, was just about the first piece of code I wrote for Python.)
  • Here are some of the issues with pgen that annoy me.
    • The “1” in the LL(1) moniker implies that it uses only a single token lookahead, and this limits our ability of writing nice grammar rules.
    • Because of the single-token lookahead, the parser cannot determine whether it is looking at the start of an expression or an assignment.
  • So how does a PEG parser solve these annoyances? By using an infinite lookahead buffer!
  • The typical implementation of a PEG parser uses something called “packrat parsing”, which not only loads the entire program in memory before parsing it, but also allows the parser to backtrack arbitrarily.
  • Why not sooner? Memory! But that is much less of an issue now.
  • My idea now, putting these things together, is to see if we can create a new parser for CPython that uses PEG and packrat parsing to construct the AST directly during parsing, thereby skipping the intermediate parse tree construction, possibly saving memory despite using an infinite lookahead buffer

Extras

Brian:

Michael:

Jokes

A couple of quick ones:

  • “What is a whale’s favorite language?” “C” — via Eric Nelson
  • Why does Pythons live on land? Because it is above C-level! — via Jesper Kjær Sørensen @JKSlonester

Jul 29 2019

30mins

Play

#140 Becoming a 10x Developer (sorta)

Podcast cover
Read more

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean

Brian #1: Becoming a 10x Developer : 10 ways to be a better teammate

  • Kate Heddleston
  • “A 10x engineer isn’t someone who is 10x better than those around them, but someone who makes those around them 10x better.”
    1. Create an environment of psychological safety
    2. Encourage everyone to participate equally
    3. Assign credit accurately and generously
    4. Amplify unheard voices in meetings
    5. Give constructive, actionable feedback and avoid personal criticism
    6. Hold yourself and others accountable
    7. Cultivate excellence in an area that is valuable to the team
    8. Educate yourself about diversity, inclusivity, and equality in the workplace
    9. Maintain a growth mindset
    10. Advocate for company policies that increase workplace equality
  • article includes lots of actionable advice on how to put these into practice.
  • examples:
    • Ask people their opinions in meetings.
    • Notice when someone else might be dominating a conversation and make room for others to speak.

Michael #2: quasar & vue.py

  • via Doug Farrell
  • Quasar is a Vue.js based framework, which allows you as a web developer to quickly create responsive++ websites/apps in many flavours:
    • SPAs (Single Page App)
    • SSR (Server-side Rendered App) (+ optional PWA client takeover)
    • PWAs (Progressive Web App)
    • Mobile Apps (Android, iOS, …) through Apache Cordova
    • Multi-platform Desktop Apps (using Electron)
  • Great for python backends
  • tons of vue components
  • But could it be all python?
    • vue.py provides Python bindings for Vue.js. It uses brython to run Python in the browser.
    • Examples can be found here.

Brian #3: Regular Expressions 101

  • We talked about regular expressions in episode 138
  • Some tools shared with me after I shared a regex joke on twitter, including this one.
  • build expressions for Python and also PHP, JavaScript, and Go
  • put in an example, and build the regex to match
  • explanations included
  • match information including match groups and multiple matches
  • quick reference of all the special characters and what they mean
  • generates code for you to see how to use it in Python
  • Also fun (and shared from twitter):
    • Regex Golf
      • see how far you can get matching strings on the left but not the list on the right.
        • I got 3 in and got stuck. seems I need to practice some more

Michael #4: python-diskcache

  • Caching can be HUGE for perf benefits
  • But memory can be an issue
  • Persistence across executions (e.g. web app redeploy) an issue
  • Servers can be issues themselves
  • Enter the disk! Python disk-backed cache (Django-compatible). Faster than Redis and Memcached. Pure-Python.
  • DigitalOcean and many hosts now offer SSD’s be default
  • Unfortunately the file-based cache in Django is essentially broken.
  • DiskCache efficiently makes gigabytes of storage space available for caching.
    • By leveraging rock-solid database libraries and memory-mapped files, cache performance can match and exceed industry-standard solutions.
    • There's no need for a C compiler or running another process.
    • Performance is a feature
    • Testing has 100% coverage with unit tests and hours of stress.
  • Nice comparison chart

Brian #5: The Python Help System

  • Overview of the built in Python help system, help()
  • examples to try in a repl
    • help(print)
    • help(dict)
    • help('assert')
    • import math; help(math.log)
  • Also returns docstrings from your non-built-in stuff, like your own methods.

Michael #6: Python Architecture Graphs

  • by David Seddon
  • Impulse - a CLI which allows you to quickly see a picture of the import graph any installed Python package at any level within the package.
  • Useful to run on an unfamiliar part of a code base, to help get a quick idea of the structure.
  • It's a visual explorer to give you a quick signal on architecture.
  • Import Linter - this allows you to declare and check contracts about your dependency graph, which gives you the ability to lint your code base against architectural rules.
  • Helpful to enforce certain architectural constraints and prevent circular dependencies creeping in.

Extras

Michael:

Jokes

Two threads walk into a bar. The barkeeper looks up and yells, 'Hey, I want don't any conditions race like time last!’

A string value walked into a bar, and then was sent to stdout.

Jul 23 2019

24mins

Play