Cover image of Python Bytes
(109)

Rank #76 in Technology category

Technology

Python Bytes

Updated 4 days ago

Rank #76 in Technology category

Technology
Read more

Python Bytes is a weekly podcast hosted by Michael Kennedy and Brian Okken. The show is a short discussion on the headlines and noteworthy news in the Python, developer, and data science space.

Read more

Python Bytes is a weekly podcast hosted by Michael Kennedy and Brian Okken. The show is a short discussion on the headlines and noteworthy news in the Python, developer, and data science space.

iTunes Ratings

109 Ratings
Average Ratings
99
4
3
2
1

Great Python info

By alltheothernamesweretaken54136 - Dec 18 2018
Read more
Great way to keep up with the goings-on in Python.

Perfect for the morning commute.

By DangerFarr - Sep 13 2018
Read more
A quick & efficient way to keep up with news and development in the Python space!

iTunes Ratings

109 Ratings
Average Ratings
99
4
3
2
1

Great Python info

By alltheothernamesweretaken54136 - Dec 18 2018
Read more
Great way to keep up with the goings-on in Python.

Perfect for the morning commute.

By DangerFarr - Sep 13 2018
Read more
A quick & efficient way to keep up with news and development in the Python space!
Cover image of Python Bytes

Python Bytes

Updated 4 days ago

Rank #76 in Technology category

Read more

Python Bytes is a weekly podcast hosted by Michael Kennedy and Brian Okken. The show is a short discussion on the headlines and noteworthy news in the Python, developer, and data science space.

Rank #1: #9 Walking with async coroutines, diving deep into requests, and a universe of options (for AIs)

Podcast cover
Read more

This is Python Bytes, Python headlines and news deliver directly to your earbuds: episode 9, recorded on Tuesday, January 17th. In this episode we discuss walking with async coroutines, diving deep into requests, and a universe of options (for AIs).

#1 (Brian): Python Asynchronous I/O Walkthrough

  • In July, there was an open source book published called 500 Lines or Less.
  • One of the chapters was called A Web Crawler With asyncio Coroutines, written by A. Jesse Jiryu Davis and Guido van Rossum. It explains async networking, showing how non-blocking sockets work and how Python 3’s coroutines improve asynchronous network programs.
  • As mentioned recently on A. Jesse Jiryu Davis’s blog emptysqua.re, the chapter may be difficult for a novice or intermediate developer to follow.
  • This month, Philip Guo, an Assistant Professor at UC San Diego, has released an 8 part video series, 90 minutes total, so on the order of 10 minutes each, that walks through the article, including some coding examples that he walks through.
  • Since async is something that takes a while to get your head around, I appreciate the multi-sensory education experience. Listening to Philip talk about it, watching him walk through the code and talk about the article, and having the article as a reference, is super helpful.
  • Talk Python #22: CPython Internals and Learning Python with pythontutor.com
  • Talk Python #69: Write an Excellent Programming Blog

#2 (Michael): 4 likely future twists for Python
by Serdar Yegulalp

  1. Python 2.x may live on

    • Python 2.x might also get a continued lease on life if independent developers decide to keep the branch going on their own.
    • At least one such effort exists -- Naftali Harris's "Python 2.8" project, which backports improvements and bug fixes from Python 3 into the Python 2.x branch.
    • it makes sense to make the 3.x leap, but it's likely we'll see a lot of keep-the-2.x-flame-alive efforts
  2. Requirements.txt may be replaced with something better

    • Pipfile has been proposed as a possible replacement by the folks at the Python Packaging Authority, which is "the working group that maintains many of the relevant projects in Python packaging."
    • Pipfile will be superior to requirements.txt file in a number of ways:
      1. TOML syntax for declaring all types of Python dependencies.
      2. One Pipfile (as opposed to multiple requirements.txt files).
      3. Existing requirements files tend to proliferate into multiple files - e.g. dev-requirements.txt, test-requirements.txt, etc. - but a Pipfile will allow seamlessly specifying groups of dependencies in one place.
      4. This will be surfaced as only two built-in groups (default & development). (see note below)
      5. Fully specified (and deterministic) environments in the form of Pipfile.freeze. A deployed application can then be completely redeployed with the same exact versions of all recursive dependencies, by referencing the Pipfile.freeze file.
    • Example pipfile:

      [[source]]
      url = 'https://pypi.org/'
      verify_ssl = true

      [requires]
      python_version = '2.7'

      [packages]
      requests = { extras = ['socks'] }
      Django = '>1.10'
      pinax = { git = 'git://github.com/pinax/pinax.git', ref = '1.4', editable = true }

      [dev-packages]
      nose = '*'

  3. Python could get more enterprise editions

    • As the language has gained traction across the board, it's also appearing in versions aimed specifically at solving enterprise-grade problems.
    • Intel, for instance, elected to repackage the Anaconda science-and-math distribution of Python after outfitting it with extensions that give it a speed boost, albeit only on Intel processors.
    • Anaconda is itself produced by Continuum Analytics, no stranger to enterprise data-analysis needs.
  4. Python's new software repository system could lead to enterprise-friendly Python package management
    • One possibility being floated in this vein is the concept of an enterprise-grade package index for Python, as discussed by Cristian Medina of Nimble Storage:
    • Businesses always have a need for an on-premises, secure, encrypted and highly available distribution mechanism of compiled binaries. Together with setuptools providing various install capabilities that can cover non-Python code just as well, it seems like we could put together a decent product. Something like a Docker Trusted Registry.
    • I’m working on something here too that come out of the discussion of Talk Python #84

#3 (Brian) From Python to Numpy

  • Nicolas P. Rougier - labri.fr
  • “Nicolas P. Rougier is a full-time research scientist at Inria which is the French national institute for research in computer science and control.”
  • Creative commons book for people who are intermediate level Python developers and want to use numpy for science or engineering.

#4 (Michael): openai/universe

  • Universe: a software platform for measuring and training an AI's general intelligence across the world's supply of games, websites and other applications. https://universe.openai.com
  • Universe allows anyone to train and evaluate AI agents on an extremely wide range of real-time, complex environments.
  • Universe makes it possible for any existing program to become an OpenAI Gym environment, without needing special access to the program's internals, source code, or APIs.
    • It does this by packaging the program into a Docker container
    • presenting the AI with the same interface a human uses:
    • sending keyboard
    • mouse events
    • receiving screen pixels
    • Our initial release contains over 1,000 environments in which an AI agent can take actions and gather observations.

#5: (Brian): Python requests deep dive

  • Anthony Shaw - on medium.com
  • Converted a large project from httplib to requests
  • Apache Libcloud
    • needed to provide a set of base classes that would handle HTTP and HTTPS REST/JSON, REST/XML and various other bizarre HTTP APIs.
    • Libcloud has over 80 client libraries for every major cloud service out there.
    • single Connection class that handles encoding and decoding of JSON, XML or Raw data.
  • Anthony walks through the types of changes made, including authentication, session handling, testing, prepared requests, streams, …
  • uses requests-mock
    • very cool API

#6 (Michael): What's the community favorite for developing OSX desktop applications with Python?

  • bangeron: I've been using PyQt for all my projects with a GUI: - It has a native look and feel - It's cross-platform (I do everything on macOS) - It's a whole framework, including modules for plotting charts, networking, SQL, etc. - It has excellent documentation. - It has a GUI editor (Qt Creator) that is pretty good, once you figure it out.
  • Omnius: Same but using pyside to avoid license fees.
  • spinwizard69: Use TK for simple apps! It is included with Python and avoids dependency hell.
  • EssaAlshammri: Take a look at Toga its idea is great.
  • Ruditorres: Kivy hands down, it is pythonic, easy, and beautiful
  • bangeron: Kivy is anything but beautiful. At least, not without spending some time creating your own theme.
  • Personally, I want py_electron (modeled after electronjs).
  • Notable mention, related: cx_Freeze released (Version 5.0.1 (January 2017))
Jan 17 2017
23 mins
Play

Rank #2: #10 Dismissing Python's Garbage Collection, PyPI Name Reservations, and Hackers Exfiltrate US Government Data to Save Itself

Podcast cover
Read more

This is Python Bytes, Python headlines and news deliver directly to your earbuds: episode 10, recorded on Monday, January 23rd, 2016.

#1 (Brian): Advanced Time Series Plots in Python from blackarbs.com

  • matplotlib 2.0 official release on Jan 17
  • A tutorial for using pandas, numpy, matplotlib, and seaborn for collecting, analyzing, and plotting time series plots.
    • start with an empty xy chart
    • grab some yahoo finance data, do some manipulation, and plot it.
    • add shaded areas, in the example, recession times
    • chart titles
    • axis labels
    • legend styling
    • horizontal line, at the 0 value, in this example
    • formatting x and y tick labels
    • controlling the font
    • turning on data points by specifying a marker shape
    • add some chart annotations with arrows pointing to specific data points
    • add a logo and watermark over the top of everything
  • seaborn is a statistical data visualization tool for use with matplotlib
  • See the seaborn site has tutorials for:
    • Style functions, Color palettes, Distribution plots, Regression plots, Categorical plots, and Axis grid objects.

#2 (Michael): Dismissing Python Garbage Collection at Instagram

  • By dismissing the Python garbage collection (GC) mechanism, which reclaims memory by collecting and freeing unused data, Instagram can run 10% more efficiently.
  • How We Run Our Web Server: Instagram’s web server runs on Django in a multi-process mode with a master process that forks itself to create dozens of worker processes that take incoming user requests. For the application server, we use uWSGI with pre-fork mode to leverage memory sharing between master and worker processes.
  • Attempt 1: Disable reference count on code objects (no effect)
  • Attempt 2: Let’s try disabling GC
    • With that, we successfully raised the shared memory of each worker process from 140MB to 225MB, and the total memory usage on the host dropped by 8GB per machine. This saved 25% RAM for the whole Django fleet. With such big head room, we're capable of running a lot more processes or running with a much higher RSS memory threshold. In effect, this improves the throughput of Django tier by more than 10%.
  • Attempt 3: Completely shutdown GC takes churns (bad)
  • Attempt 4: Final step for shutting down GC: No cleanup (success)

#3 (Brian): My experience with type hints and mypy

Brett Cannon on his blog snarky.ca

  • Here's an example including the pull request of changes, of a project adding static types and static analysis of those types.
  • Trying to use type hints to increase code quality and readability for a project.
  • The project is CLA enforcement bot for the PSF, to make sure that all pull requests contributions to a python project is done by someone who has signed a PSF Contributor Licence Agreement.
  • Process
    • Run mypy against your code with no types, then slowly add types, one object at a time.
    • mypy is a static analysis tool to make sure your typehints are correct.
  • Issues:
    • supports Python 3.5, but not new types in Python 3.6, specifically typying. Collection. Workaround was to use typing.AbstractSet
    • f-strings not supported yet, but coming
    • Skipped type hinting for test suite.
    • It wouldn't occur to me to even try to add static types to tests.
    • Most linters would have caught the problems as well as mypy.
  • Benefit
    • The act of adding static types increased the readability of the code. Did not detract from readability.
    • “What mypy really got me was better documentation. While I was adding the type hints there were a couple of times where I had to examine the code to realize what the appropriate type was. Now that I have the functions and methods all hinted I don't have to guess anymore. That should make long-term maintenance a bit easier”
  • Conclusion
    "…would I bother typing new Python 3 code? … yes once mypy supports f-strings.
    “When I design an API I already have to think about what type of objects would be acceptable, so quickly writing down my assumptions doesn't hurt anything, it's relatively quick, and it benefits anyone having to work with my code. But I also wouldn't contort my code to fit within the confines of type hints (i.e. if type hints forces me to write cleaner code then that's great, but if something is so dynamic that it can't have type hints then that's fine and I'll happily use typing.Any as an escape hatch). “
    “In the end I view type hints as enhanced documentation that has tooling to help verify that the documentation about types is accurate. And for that use-case I see type hints worth doing and not at all a burden.”

#4 (Michael): Understanding the underscore( _ ) of Python

  • The underscore (_) is special in Python.
  • If you are python programmer, for _ in range(10) , __init__(self) like syntax may be familiar.
  • There are 5 cases for using the underscore in Python.
    • For storing the value of last expression in interpreter.
    • For ignoring the specific values. (so-called “I don’t care”)
    • To give special meanings and functions to name of vartiables or functions.
    • _protected-ish, __private-ish, conflicting: in_, __magic__
    • To use as ‘Internationalization(i18n)’ or ‘Localization(l10n)’ functions.
    • To separate the digits of number literal value.

#5: (Brian): PEP 541 -- Package Index Name Retention
Draft status currently; was just submitted on the 12th by Donald Stufft.
“ Abstract:
This PEP proposes an extension to the Terms of Use of the Package Index, clarifying expectations of package owners regarding ownership of a package name on the Package Index, specifically with regards to conflict resolution.

Existing package repositories such as CPAN , NPM , and GitHub will be investigated as prior art in this field.

Rationale
Given that package names on the Index are sharing a single flat namespace, a unique name is a finite resource. The growing age of the Package Index causes a constant rise of situations of conflict between the current use of the name and a different suggested use of the same name.

This document aims to provide general guidelines for solving the most typical cases of such conflicts.”

#6 (Michael): Hackers downloaded US government climate data and stored it on European servers as Trump was being inaugurated

  • As Donald Trump was sworn into office as the new president of the US on Jan. 20, a group of around 60 programmers and scientists were gathered in the Department of Information Studies building at the University of California-Los Angeles, harvesting government data.
  • A spreadsheet detailed their targets:
    • Webpages dedicated to the Department of Energy’s solar power initiative
    • Energy Information Administration data sets that compared fossil fuels to renewable energy sources
    • fuel cell research from the National Renewable Energy Laboratory, to name a few out of hundreds.
  • Volunteer “data rescue” events in Toronto, Philadelphia, Chicago, Indianapolis, and Michigan over the last few weeks have managed to scrape hundreds of thousands of pages off of EPA.gov, NASA.gov, DOE.gov, and whitehouse.gov, uploading them to the Internet Archive. Another is planned for early February at New York University.
  • Suddenly, at exactly noon on Friday as Trump was sworn in, and just as the UCLA event kicked off, some of their fears began to come true: The climate change-related pages on whitehouse.gov disappeared.
  • There will, thanks to Michael Riedyk, CEO of the Canadian data-archiving company Page Freezer, also be a copy stored outside the US.
Jan 23 2017
25 mins
Play

Rank #3: #14 Lots of Python style and Python 3000 is 3000 days old

Podcast cover
Read more

Brian #1: Tiny Python 3.6 Notebook - Matt Harrison

Michael #2: Oh no! This package is Python 2 only - Anthony Shaw

  • You’re head down, working on a new project and one of your dependencies still doesn’t support Python 3 - argh! Here’s a quick guide on how to solve that problem, step by step.
  • Step 0: Check that nobody else has solved this
  • Step 1: Fork it
  • Step 2: Print statements
  • Step 3: Tests
    • Static Analysis: modernize
  • Step 4: Update setup.py
  • Step 5: Install it into your original project
  • Step 6: Raise a pull request
  • Step 7: 3 months later?

Brian #3: Elements of Python Style - Andrew Montalenti

  • More than PEP8, and opinionated.
  • I admire the effort, and I may at some point fork it to remove/fix the few things I disagree with and to be able to use it as a style guide for my team.
  • Great: most of it.
    • Use parens for continuation
    • Use with for files and locks
  • May need tweaking
    • Avoid custom exceptions
    • I’d prefer, “Add one custom base exception, and build specific exceptions off of that”.
    • This may be a controversial point, or I may be just confused about conventional practice here
    • As with any style guide, more detail brings more controversy.
  • Michael: I like this guidance: You can also choose to use CamelCase for things that are class-like but not quite classes -- the main benefit of CamelCase is calling attention to something as a "global noun", rather than a local label or a verb. Notice that Python names True, False, and None use CamelCase even though they are not classes.

Michael #4: Python 3 was exactly 3000 days old this past Sunday

  • Reddit post about my tweet (wow)
    • 537 upvotes, 71 comments
  • Test it yourself:
    background =
    “Python 3 was released December 3, 2008 ” + \
    “Its original working name was Python 3000”
    release = datetime.date(year=2008, month=12, day=3)
    today = datetime.date(year=2017, month=2, day=19)
    (today - release).days # → 3000
  • Via @cclauss Christian Clauss

Brian #5:
From beginner to pro: Python books, videos and resources

Michael #6: mongoaudit

  • mongoaudit is a CLI tool for auditing MongoDB servers, detecting poor security settings and performing automated penetration testing.
  • pip install mongoaudit
  • Tests things like:
    • MongoDB listens on a port different to default one
    • Server only accepts connections from whitelisted hosts / networks
    • MongoDB HTTP status interface is not accessible on port 28017
    • MongoDB is not exposing its version number
    • MongoDB version is newer than 2.4
    • TLS/SSL encryption is enabled
    • Authentication is enabled
    • SCRAM-SHA-1 authentication method is enabled
    • Server-side Javascript is forbidden
    • Roles granted to the user only permit CRUD operations
    • The user has permissions over a single database
    • Various security vulnerabilities
  • From Michael: MongoDB is awesome. But please make sure at least one of the following is true.
    • You only listen on local loopback (127.0.0.1) and run mongo on the web server
    • You run mongodb with authentication enabled (it’s off by default)
    • You run mongodb with SSL enabled (may be off by default too)

What else? Launched my course at https://bit.ly/python-rest-course

Feb 22 2017
15 mins
Play

Rank #4: #4 Python 3 is just fine for beginners thank you, q is awesome for debugging, and more

Podcast cover
Read more

This is Python Bytes, Python headlines and news deliver directly to your earbuds: episode 4: recorded on November 28, 2016. In this episode we cover the case for Python 3, asyncio, pyston, pydoc.io, and q.

This episode was brought to you by Rollbar: they help you take the pain out of errors.

News items

The Case FOR Python 3

>> "It's as simple as that. If you learn Python 2, then you can still work with all the legacy Python 2 code in existence **until Python dies or you (hopefully) move on**. But if you learn Python 3 then your future is very uncertain. You could really be learning a dead language and end up having to learn Python 2 anyway." -- Zed S.
  • This is harmful to the Python community.
  • Let's stop recommending this book and stop elevating this guy's work
  • "Arguments" from Zed:
    • Not In Your Best Interests: "The Python project’s efforts to convince you to start with Python 3 are not in your best interest, but, rather, are only in the best interests of the Python project."
    • You Should Be Able to Run 2 and 3 (in the same process). The fact that no one has (yet) written a Python 3 interpreter that can simultaneously run Python 2 in the same process means Python 3 is not Turing complete
    • Difficult To Use Strings: "The strings in Python 3 are very difficult to use for beginners. In an attempt to make their strings more “international” they turned them into difficult to use types with poor error messages."
    • Final analysis:
      • You can’t add b"hello" to "hello".
      • Too many formatting choices for strings
      • Bytes aren't automatically decoded to strings
    • Therefore: All newbies should avoid Python 3 like the plague - it will curse your career and cloud your judgement.
  • Excellent rebuttal article: Rebuttal for Python 3

Reddit discussion of a Feb article on "AsyncIO for the Working Python Developer"

  • Article is a nice introduction, but uses Python 3.4 syntax
  • Interesting to note that no one in the reddit comments suggests a better 3.5 tutorial. Is async/await just too confusing for a 3.5 intro article? Or is it so simple that no one has thought it needs a tutorial.
  • 23 page article / tutorial on Python 3.4's asyncio
  • Not covering Python 3.5's async / await, but great gateway article
  • Here’s a call for action. Someone write a similar article but with 3.5+ syntax. Also could we show examples without sleep calls. Sleep is such a lame way to show “do some actual work”. Maybe use a producer/consumer or reader/writer example or something.

Pyston 0.6 released

  • Pyston is an open source Python implementation that aims to be both highly compatible and high-performance.
  • It uses modern JIT techniques and natively supports many CPython C extension modules.
  • Pyston is sponsored by Dropbox, and is pronounced "piston".
  • main goal was to reduce the overall memory footprint. It also contains a lot of additional smaller changes for improving compatibility and fixing bugs.
    • We also spent time on making it easier to replace CPython with Pyston, such as by more closely matching its directory structure
    • following its ‘dict’ ordering.
    • We can now, for example, run pip and virtualenv unmodified, without requiring any upstream patches like other implementations do.
  • Unclear how much of a research project vs an actual project this is at the moment

Announcing pydoc.io

  • Automatic API documentation from pypi repositories.
  • Pydoc.io will eventually auto-generate API reference documentation for every package on PyPI.

What one thing took your Python to the next level

  1. Mastering generators
  2. Understanding how iteration really works in Python opens up many possibilities for elegant, high performance code. Use iPython. Learn that for does tuple unpacking; play with zip, enumerate, all, any; take a look at the itertools module
  3. Unit tests! Pytest in particular! Gives you a whole new perspective on your code and dammit, it's so satisfying to get all those green OKs!
  4. list comprehensions
  5. For me, all the work of David Beazley, in particular http://www.dabeaz.com/coroutines/

q : Quick-and-dirty debugging output for tired programmers

Our personal news

Michael

Brian

Nov 29 2016
21 mins
Play

Rank #5: #141 Debugging with f-strings coming in Python 3.8

Podcast cover
Read more

Sponsored by Datadog: pythonbytes.fm/datadog

Brian #1: Debugging with f-strings in Python 3.8

  • We’ve talked about the walrus operator, :=, but not yet “debug support for f-strings”
  • this: print(f'foo={foo} bar={bar}')
  • can change to this: print(f'{foo=} {bar=}')
  • and if you don’t want to print with repr() you can have str() be used with !s.
    • print(f'{foo=!s} {bar=!s}')
  • also !f can be used for float modifiers:
>>> import math
>>> print(f'{math.pi=!f:.2f}')
math.pi=3.14
  • one more feature, space preservation in the f-string expressions:
>>> a = 37
>>> print(f'{a = }, {a = }')
a = 37, a = 37

Michael #2: Am I "real" software developer yet?

  • by Sun-Li Beatteay
  • To new programmers joining the field, especially those without CS degrees, it can feel like the title is safe-guarded. Only bestowed on the select that have proven themselves.
  • Sometimes manifests itself as Impostor Syndrome
  • Focused on front-end development as I had heard that HTML, CSS and JavaScript were easy to pick up
  • That was when I decided to create a portfolio site for my wife, who was a product designer.
  • Did my best to surround myself with tech culture.
    • Watched YouTube videos
    • listened to podcasts
    • read blog posts from experienced engineers to keep myself motivated.
    • Daydreamed what it would be like to stand in their shoes.
  • My wife’s website went live in July of that year. I had done it.
  • Could I finally start calling myself something of a Software Engineer?
    • “Web development isn’t real programming”
  • Spent the next 18 months studying software development full time. I quit my job and moved in with my in-laws — which was a journey in-and-of itself.
    • Software engineer after 1-2 years? No so fast (says the internet)
  • The solution that I found for myself was simple yet terrifying: talking to people
  • MK: BTW, I don’t really like the term “engineer”

Brian #3: Debugging with local variables and snoop

  • debugging tools
  • ex: “You want to know which lines are running and which aren't, and what the values of the local variables are.”
    • Throw a @snoop decorator on a function and the function lines and local variable values will be dumped to stderr during run. Even showing loops a bunch of times.
  • It’s tools to almost debug as if you had a debugger, without a debugger, and without having to add a bunch of logging or print statements.
  • Lots of other use models to allow more focus.
    • wrap just part of your function with a with snoop block
    • only watch certain local variables.
    • turn off reporting for deep function/block levels.

Michael #4: New home for Humans

  • This came out of the blue with some trepidation:
  • kennethreitz commented 6 days ago:

In the spirit of transparency, I'd like to (publicly) find a new home for my repositories. I want to be able to still make contributions to them, but no longer be considered the "owner" or "arbiter" or "BDFL" of these repositories.

Some notable repos:

Brian #5: The Backwards Commercial License

  • Eran Hammer - open source dev, including hapi.js
  • Interesting idea to make open source projects maintainable
  • Three phases of software lifecycle for some projects:
    • first: project created to fill a need in one project/team/company, a single use case
    • second: used by many, active community, growing audience
    • three: work feels finished. bug fixes, security issues, minor features continue, but most people can stay on old stable versions
  • During the “done” phase, companies would like to have bug fixes but don’t want to have to keep changing their code to keep up.
  • Idea: commercial license to support old stable versions.
    • “If you keep up with the latest version, you do not require a license (unless you want the additional benefits it will provide).”
    • “However, very few companies can quickly migrate every time there is a new major release of a core component. Engineering resources are limited and in most cases, are better directed at building great products than upgrading supporting infrastructure. The backwards license provides this exact assurance. You can stay on any version you would like knowing that you are still running supported, well-maintained, and secure code.”
    • “The new commercial license will include additional benefits focused on providing enterprise customers the assurances needed to rely on these critical components for many years to come. “

Michael #6: Switching Python Parsers?

  • via Gi Bi, article by Guido van Rossum
  • Alternative to the home-grown parser generator that I developed 30 years ago when I started working on Python. (That parser generator, dubbed “pgen”, was just about the first piece of code I wrote for Python.)
  • Here are some of the issues with pgen that annoy me.
    • The “1” in the LL(1) moniker implies that it uses only a single token lookahead, and this limits our ability of writing nice grammar rules.
    • Because of the single-token lookahead, the parser cannot determine whether it is looking at the start of an expression or an assignment.
  • So how does a PEG parser solve these annoyances? By using an infinite lookahead buffer!
  • The typical implementation of a PEG parser uses something called “packrat parsing”, which not only loads the entire program in memory before parsing it, but also allows the parser to backtrack arbitrarily.
  • Why not sooner? Memory! But that is much less of an issue now.
  • My idea now, putting these things together, is to see if we can create a new parser for CPython that uses PEG and packrat parsing to construct the AST directly during parsing, thereby skipping the intermediate parse tree construction, possibly saving memory despite using an infinite lookahead buffer

Extras

Brian:

Michael:

Jokes

A couple of quick ones:

  • “What is a whale’s favorite language?” “C” — via Eric Nelson
  • Why does Pythons live on land? Because it is above C-level! — via Jesper Kjær Sørensen @JKSlonester
Jul 29 2019
30 mins
Play

Rank #6: #63 We're still on a desktop GUI kick

Podcast cover
Read more

Sponsored by DigitalOcean: http://do.co/python

Brian #1: A brief tour of Python 3.7 data classes

  • a great write-up of the upcoming data classes via Anthony Shaw
  • “Data classes are a way of automating the generation of boiler-plate code for classes which store multiple properties. They also carry the benefit of using Python 3’s new type hinting.”
  • Default magic methods
    • In the default setting, any dataclass will implement __init__, __repr__, __str__ and __eq__ for you.
    • The __init__ method will have keyword-arguments with the same type annotations that are specified on the class.
    • The __eq__ method will compare all dataclass attributes in order.
    • All fields are declared at the top of the class and type hinting is required.
  • Also covered
    • type hinting
    • mutability (and frozen)
    • customizing the fields
    • post-init processing : optional __``*post_init_*``_ will run after the generated _``*_init_*``_
    • inheritance

Michael #2: SQLite [The Databaseology Lectures - CMU Fall 2015]

Brian #3: dryable : a useful dry-run decorator for python

  • short circuit methods within your project during dry runs.
  • example shows how to add a command line flag --dry-run.
  • The test code is useful for understanding it also.
  • Example something.py
    import dryable

    @dryable.Dryable('foo')
    def return_something():
    return 'something'

test_something.py

from something import return_something
import dryable

def test_normal_return():
dryable.set(False)
assert return_something() == 'something'

def test_dry_return(capsys):
dryable.set(True)
assert return_something() == 'foo'

Michael #4:

Brian #5: PEP Explorer - Explore Python Enhancement Proposals

  • Cool idea. Might need some work though. I can’t find any accepted PEPs for 3.7, including 557, data classes.
  • I’m ok with giving Anthony some shade on this, as we highlighted his writing in the first item.

Michael #6: TKInter Tutorial

  • via @likegeeks
  • Create your first GUI application
  • Create a label and button widgets
  • Input and combo boxs, menus, progressbars and more

Our news

Michael

  • I built something with Gooey this weekend, it was wonderful.
  • Self-serve team purchases and discounts at Talk Python Training
Feb 01 2018
21 mins
Play

Rank #7: #64 The GUI phoenix rises with wxPython

Podcast cover
Read more

Sponsored by DigitalOcean: http://do.co/python

Brian #1: wxPython 4, Pheonix is now live and supports Python 3

Michael #2: typeshed

  • Typeshed contains external type annotations for the Python standard library and Python builtins, as well as third party packages.
  • This data can e.g. be used for static analysis, type checking or type inference.
  • Used as the basis of mypy and PyCharm’s magic
  • Each Python module is represented by a .pyi "stub". This is a normal Python file (i.e., it can be interpreted by Python 3), except all the methods are empty. Python function annotations (PEP 3107) are used to describe the types the function has.
  • Here’s what one of these exeternal definitions looks like:
class NodeVisitor():
def visit(self, node: AST) -> Any: ...
def generic_visit(self, node: AST) -> None: ...

Brian #3: Coverage 4.5 adds configurator plug-ins

  • “There’s one new feature: configurator plug-ins, that let you run Python code at startup to set the configuration for coverage. This side-steps a requested feature to have different exclusion pragmas for different versions of Python.”

Michael #4: Python integrated into Unreal Engine

  • via Pirogov Alexander‏ ( @Pie_Daddy )
  • tl;dr: Autodesk university plans to integrate Python into Unreal Engine for the data integration pipeline and ease the process of bringing assets into the game.
  • Autodesk is working on that will solve complicated problems with bringing CAD data into the Unreal Engine.
  • Where they are today:
    • The Datasmith workflow toolkit, currently in beta, makes moving data into Unreal Engine as frictionless as possible.
    • Datasmith provides high-fidelity translation of common scene assets such as geometry, textures, materials, lights and cameras from popular DCC and CAD applications into Unreal Engine.

Brian #5: Python 3.7.0b1 : Beta means we should be testing it!!!

  • If not people like us and our listeners, then who? Seems like we’re a good set of beta testers.
  • What are you going to test?
  • I'm going to look at breakpoint() and data classes.

Michael #6: Releases abound!

Our news

Brian:

  • Speaking at PyCon 2018. “PyCharm and pytest”. Speaking with Paul Everitt
  • Upcoming webinar: Productive pytest with Pycharm
    • Feb 22, registration open

Michael:

Feb 09 2018
21 mins
Play

Rank #8: #45 A really small web API and OS-level machine learning

Podcast cover
Read more

This episode is brought to you by Rollbar: pythonbytes.fm/rollbar

Brian #1: pico

  • "a very small web application framework for Python"
  • Recommended by Ivan Pejić
  • lightning talk from EuroPython 2017
  • This would be a good web framework for building internal services and tools that non-web developers need to interact with and modify.
  • Very simple.
  • Not REST, but not confusing either.

Michael #2: High Sierra ships, first major OS with machine learning built in?

  • September 26th macOS High Sierra was released (yay)
  • Mostly a foundational release with barely visible changes but awesome things like APFS replacing HFS+, etc.
  • Comes with CoreML
    • Apple’s intent with the new CoreML framework is to package up prebuilt ML models and execution engines and make them possible for third-party apps to use.
    • Developers can take a trained machine learning algorithm, package it up as an MLModel, and integrate it into their apps.
    • Apple offers a few default machine learning models that developers can download and use too
  • Rather than sharing your data with a central server, grouping it together with a lot of other people's data, and improving machine learning models that way, Apple stresses that everything CoreML does is happening on the device.
  • On Macs that support Metal—generally, Macs from 2012 and later—CoreML uses a mix of CPU processing and GPGPU processing, depending on the task.
  • Add on the fact that High Sierra has external GPU support now and you have a sweet combo.

Brian #3: A guide to logging in Python

  • Simply put, the best logging introduction I've read so far.

Michael #4: Let me introduce: slots

  • So what are slots? __slots__ are a different way to define the attributes storage for classes in Python.
  • for normal Python classes, a dict is used to store the instance's attributes.
  • With __slots__ we don't have an attribute called __dict__ inside our instance. But we have a new attribute called __slots__.
  • But why would you need to use slots when you have a dict? Well the answer is that __slots__ are a lot lighter and slightly faster.
  • Outcome:
    • ~57% less memory usage thanks to just one line of code.
    • __slots__ are also slightly faster.
  • Covered in depth in my Write Pythonic Code Like a Seasoned Developer course.

Brian #5: pipenv revisited

  • Covered in episode 11. However, there are some notable changes since then.
  • Reminder:
    • pepenv handles virtualenv and pip interaction for you
    • pipenv install creates a virtualenv (if one doesn't exist) and installs stuff into a virtualenv.
    • pipenv shell uses the virtualenv
    • exit allows you to get out of the virtualenv
    • pipenv lock -r will generate a requirements.txt file for you, so you can use it even if you need a requirements.txt file.
  • Notable changes:
    • New 4 minute screencast with Kenneth demonstrating how to use it. Watching him use it makes it very simple to understand.
    • Specify multiple package indexes, and even specify a particular index for particular packages. So you can combine both pypi with a company index and a group index and maybe one for your project.
    • pipenv check will tell you about any known security vulnerabilities in your installed packages
    • 9 months old with 192 releases, so keep an eye on it for new features all the time.

Michael #6: Stack Overflow gives an even closer look at developer salaries

  • Tabs and spaces aren't the only things that influence developer pay…
  • Some of the broad trends are no big surprise; for example, the chosen cities tend to pay more than their respective nations do, for example.
  • DevOps specialists and data scientists both earn well.
  • Other aspects of the data are a little more surprising. Graphics programmers, for example, aren't particularly well paid, in spite of having a relatively specialized, complex niche.
  • And while earnings in four of the countries are surprisingly similar, those in America are substantially higher, regardless of experience; in fact, the median salary of a developer in the US is comparable to that of someone with 20 years of experience in Canada or Germany and markedly higher than 20-year veterans in France and the UK. Even after taking into account the US' higher healthcare costs, America is the place to be if you're a programmer.
  • Comments
    • I do have to wonder how much Silicon Valley skews that salary chart, as the Web 2.0 companies pay HUGE comparatively [ref]
    • I asked Stack Overflow's data scientist that question, and she said not much; even without its outlier cities, the US pays much more than the rest of the world. [ref]
    • Healthcare cost are only part of it. I got paid about $600/month 9 months a year by my government to study in university. [ref]
    • I feel like a lot of IT people lack soft skills, and it caps their salary at a lower end. [ref]

Our news:

Sep 29 2017
19 mins
Play

Rank #9: #112 Don't use the greater than sign in programming

Podcast cover
Read more

Sponsored by https://pythonbytes.fm/datadog

Brian #1: nbgrader

  • nbgrader: A Tool for Creating and Grading Assignments in the Jupyter Notebook
    • The Journal of Open Source Education, paper accepted 6-Jan-2019
  • nbgrader documentation, including a intro video
  • From the JOSE article:
    • “nbgrader is a flexible tool for creating and grading assignments in the Jupyter Notebook (Kluyver et al., 2016). nbgrader allows instructors to create a single, master copy of an assignment, including tests and canonical solutions. From the master copy, a student version is generated without the solutions, thus obviating the need to maintain two separate versions. nbgrader also automatically grades submitted assignments by executing the notebooks and storing the results of the tests in a database. After auto-grading, instructors can manually grade free responses and provide partial credit using the formgrader Jupyter Notebook extension. Finally, instructors can use nbgrader to leave personalized feedback for each student’s submission, including comments as well as detailed error information.”
  • CS teaching methods have come a long ways since I was turning in floppies and code printouts.
Michael #2: profanity-check

  • A fast, robust Python library to check for offensive language in strings.
  • profanity-check uses a linear SVM model trained on 200k human-labeled samples of clean and profane text strings.
  • Making profanity-check both robust and extremely performant
  • Other libraries like profanity-filter use more sophisticated methods that are much more accurate but at the cost of performance.
    • profanity-filter runs in 13,000ms vs 24ms for profanity-check in a benchmark
  • Two ways to use:
    • predict(text) → 0 or 1 (1 = bad)
    • predict_prob(text) → [0, 1] confidence interval (1 = bad)
Brian #3: An Introduction to Python Packages for Absolute Beginners

  • Ever tried to explain the difference between module and package? Between package-in-the-directory-with-init sense and package-you-can-distribute-and-install-with-pip sense? Here’s the article to read beforehand.
  • Modules, packages, using packages, installing, importing, and more.
  • And that’s not even getting into flit and poetry, etc. But it’s a good place to start for people new to Python.
Michael #4: Python Dependencies and IoC

  • via Joscha Götzer
  • Open-closed principle is at work with these and is super valuable to testing (one of the SOLID principles): Software entities (classes, modules, functions, etc.) should be open for extension, but closed for modification.
  • There is a huge debate around why Python doesn’t need DI or Inversion of Control (IoC), and a quick stackoverflow search yields multiple results along the lines of “python is a scripting language and dynamic enough so that DI/IoC makes no sense”. However, especially in large projects it might reduce the cognitive load and decoupling of individual components
  • Dependency Injector: I couldn’t get this one to work on windows, as it needs to compile some C libraries and some Visual Studio tooling was missing that I couldn’t really install properly. The library looks quite promising though, but sort of static with heavy usage of containers and not necessarily pythonic.
  • Injector: The library that above mentioned article talks about, a little Java-esque
  • pinject: Has been unmaintained for about 5 years, and only recently got new attention from some open source people who try to port it to python3. A product under Google copyright, and looks quite nice despite the lack of python3 bindings. Probably the most feature-rich of the listed libraries.
  • python-inject: I discovered that one while writing this email, not really sure if it’s any good. Nice use of type annotations and testing features
  • di-py: Only works up to python 3.4, so I’ve also never tried it (I’m one of those legacy python haters, I’m sure you can relate 😄).
  • Serum: This one is a little too explicit to my mind. It makes heavy use of context managers (literally with Context(...): everywhere 😉) and I’m not immediately sure how to work with it. In this way, it is quite powerful though. Interesting use of class decorators.
  • And now on to my favorite and a repeated recommendation of mine around the internet→ Haps: This lesser-known, lightweight library is sort of the new kid on the block, and really simple to use. As some of the other libraries, it uses type annotations to determine the kind of object it is supposed to instantiate, and automatically discovers the required files in your project folder. Haps is very pythonic and fits into apps of any size, helping to ensure modularization as the only dependency of your modules will be one of the types provided by the library. Pretty good example here.
Brian #5: A Gentle Introduction to Pandas

  • Really a gentle introduction to the Pandas data structures Series and DataFrame.
  • Very gentle, with console examples.
  • Create series objects:
    • from an array
    • from an array, and change the indexing
    • from a dictionaries
    • from a scalar, cool. didn’t know you could do that
  • Accessing elements in a series
  • DataFrames
    • sorting, slicing
    • selecting by label, position
    • statistics on columns
    • importing and exporting data
Michael #6: Don't use the greater than sign in programming

  • One simple thing that comes up time and time again is the use of the greater than sign as part of a conditional while programming. Removing it cleans up code.
  • Let's say that I want to check that something is between 5 and 10.
  • There are many ways I can do this
x > 5 and 10 > x
5 < x and 10 > x
x > 5 and x < 10
10 < x and x < 5
x < 10 and x > 5
x < 10 and 5 < x
  • Sorry, one of those is incorrect. Go ahead and find out which one
  • If you remove the use of the greater than sign then only 2 options remain
    • x < 10 and 5 < x
    • 5 < x and x < 10
    • The last is nice because x is literally between 5 and 10
  • There is also a nice way of expressing that "x is outside the limits of 5 and 10”
    • x < 5 or 10 < x
    • Again, this expresses it nicely because x is literally outside of 5 to 10.
  • Interesting comment: What is cleaner or easier to read comes down to personal taste. But how to express "all numbers greater than 1" without '>'?
    • ans: 1 < allNumbers
Extras

Michael

Joke: Harry Potter Parser Tongue via Nick Spirit

Jan 11 2019
28 mins
Play

Rank #10: #102 Structure of a Flask Project

Podcast cover
Read more

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean

Brian #1: QuantEcon

  • “Open source code for economic modeling”
  • “QuantEcon is a NumFOCUS fiscally sponsored project dedicated to development and documentation of modern open source computational tools for economics, econometrics, and decision making.”
  • Educational resource that includes:
    • Lectures, workshops, and seminars
    • Cheatsheets for scientific programming in Python and Julia
    • Notebooks
    • QuantEcon.py : open source Python code library for economics

Michael #2: Structure of a Flask Project

  • Flask is very flexible, it has no certain pattern of a project folder structure. Here are some suggestions.
  • I always keep this one certain rule when writing modules and packages:
  • Candidate structure:
project/
__init__.py
models/
__init__.py users.py posts.py
...
routes/
__init__.py home.py account.py dashboard.py
...
templates/
base.html
post.html
...
services/
__init__.py google.py mail.py
  • Love it! To this, I would rename routes to views or controllers and add a viewmodels folder and viewmodels themselves.
  • Brian, see anything missing?
    • ya. tests. :)
  • Another famous folder structure is app based structure, which means things are grouped bp application
  • I (Michael) STRONGLY recommend Flask blueprints

Brian #3: Overusing lambda expressions in Python

  • lambda expressions vs defined functions
    1. They can be immediately passed around (no variable needed)
    2. They can only have a single line of code within them
    3. They return automatically
    4. They can’t have a docstring and they don’t have a name
    5. They use a different and unfamiliar syntax
  • misuses:
    • naming them. Just write a function instead
    • calling a single function with a single argument : just use that func instead
  • overuse:
    • if they get complex, even a little bit, they are hard to read
    • has to be all on one line, which reduces readibility
    • map and filter : use comprehensions instead
    • using custom lambdas instead of using operators from the operator module.

Michael #4: Asyncio in Python 3.7

  • by Cris Medina
  • The release of Python 3.7 introduced a number of changes into the async world.
    • Some may even affect you even if you don’t use asyncio.
  • New Reserved Keywords: The async and await keywords are now reserved.
    • There’s already quite a few modules broken because of this. However, the fix is easy: rename any variables and parameters.
  • Context Variables: Version 3.7 now allows the use of context variables within async tasks. If this is a new concept to you, it might be easier to picture it as global variables whose values are local to the currently running coroutines.
  • Python has similar constructs for doing this very thing across threads. However, those were not sufficient in async-world
  • New asyncio.run() function
    • With a call to asyncio.run(), we can now automatically create a loop, run a task on it, and close it when complete.
  • Simpler Task Management: Along the same lines, there’s a new asyncio.create_task() function that helps make tasks that inside the current loop, instead of having to get the loop first and calling create task on top of it.
  • Simpler Event Loop Management: The addition of asyncio.get_running_loop() will help determine the active event loop, and catch a RuntimeError if there’s no loop running.
  • Async Context Managers: Another quality-of-life improvement. We now have the asynccontextmanager() decorator for producing async context managers without the need for a class that implements __aenter__() or __aexit__().
  • Performance Improvements: Several functions are now optimized for speed, some were even reimplemented in C. Here’s the list:
    • asyncio.get_event_loop() is now 15 times faster.
    • asyncio.gather() is 15% faster.
    • asyncio.sleep() is two times faster when the delay is zero or negative.
    • asyncio.Future callback management is optimized.
    • Reduced overhead for asyncio debug mode.
  • Lots lots more

Brian #5: Giving thanks with **pip thank**

Michael #6: Getting Started With Testing in Python

  • by Anthony Shaw, 33 minutes reading time according to Instapaper
  • Automated vs. Manual Testing
  • Unit Tests vs. Integration Tests: A unit test is a smaller test, one that checks that a single component operates in the right way. A unit test helps you to isolate what is broken in your application and fix it faster.
  • Compares unittest, nose or nose2, pytest
  • Covers things like:
    • Writing Your First Test
    • Where to Write the Test
    • How to Structure a Simple Test
    • How to Write Assertions
    • Dangers of Side Effects
  • Testing in PyCharm and VS Code
  • Testing for Web Frameworks Like Django and Flask
  • Advanced Testing Scenarios
  • Even: Testing for Security Flaws in Your Application

Extras:

Oct 31 2018
26 mins
Play

Rank #11: #94 Why don't you like notebooks?

Podcast cover
Read more

Sponsored by DigialOcean -- pythonbytes.fm/digitalocean

Brian #1: Python Patterns

Michael #2: Arctic: Millions of rows a sec (time data)

  • Arctic is a high-performance datastore for numeric data. It supports Pandas, numpy arrays and pickled objects out-of-the-box, with pluggable support for other data types and optional versioning.
  • Arctic can query millions of rows per second per client, achieves ~10x compression on network bandwidth, ~10x compression on disk, and scales to hundreds of millions of rows per second per MongoDB instance.
  • Arctic has been under active development at Man AHL since 2012.
  • Super fast, some latency numbers:
    • 1xDay Data 4ms for 10k rows, vs 2,210 ms from SQL Server)
    • Tick Data 1s for 3.5 MB (Python) or 15 MB (Java) vs 15-40sec from “other tick”
  • Versioned data
  • Built on MongoDB
  • Slides
  • Based on pandas
  • Tested with pytest

Brian #3: PyCon Australia videos

Michael #4: GAE: Introducing App Engine Second Generation runtimes and Python 3.7

  • Today, Google Cloud is announcing the availability of Second Generation App Engine standard runtimes, a significant upgrade to the platform that allows you to easily run web apps using up-to-date versions of popular languages, frameworks and libraries.
  • Python 3.7 is one of the new Second Generation runtimes that we announced at Cloud Next.
  • Based on technology from the gVisor container sandbox, these Second Generation runtimes eliminate many previous App Engine restrictions, giving you the ability to write portable web apps and microservices that take advantage of App Engine's unique auto-scaling, built-in security and pay-per-use billing model.
  • This new runtime allows you to take advantage of Python's vibrant ecosystem of open-source libraries and frameworks. While the Python 2 runtime only allowed the use of specific versions of whitelisted libraries, Python 3 supports arbitrary third-party libraries, including those that rely on C code and native extensions. Just add Django 2.0, NumPy, scikit-learn or your library of choice to a requirements.txt file. App Engine will install these libraries in the cloud when you deploy your app.

Brian #5: I don’t like notebooks

Michael #6: PEP 8000 -- Python Language Governance Proposal Overview

  • This PEP provides an overview of the selection process for a new model of Python language governance in the wake of Guido's retirement. Once the governance model is selected, it will be codified in PEP 13.
  • PEPs in the lower 8000s describe the general process for selecting a governance model.
    • PEP 8001 - Python Governance Voting Process
    • PEP 8002 - Open Source Governance Survey
  • PEPs in the 8010s describe the actual proposals for Python governance.
    • PEP 8010 - The BDFL Governance Model
    • PEP 8011 - The Council Governance Model
    • PEP 8012 - The Community Governance Model

Extras

Sep 06 2018
23 mins
Play

Rank #12: #108 Spilled data? Call the PyJanitor

Podcast cover
Read more

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean

Brian #1: pyjanitor - for cleaning data

  • originally a port of an R package called janitor, now much more.
  • “pyjanitor’s etymology has a two-fold relationship to “cleanliness”. Firstly, it’s about extending Pandas with convenient data cleaning routines. Secondly, it’s about providing a cleaner, method-chaining, verb-based API for common pandas routines.”
  • functionality:
    • Cleaning columns name (multi-indexes are possible!)
    • Removing empty rows and columns
    • Identifying duplicate entries
    • Encoding columns as categorical
    • Splitting your data into features and targets (for machine learning)
    • Adding, removing, and renaming columns
    • Coalesce multiple columns into a single column
    • Convert excel date (serial format) into a Python datetime format
    • Expand a single column that has delimited, categorical values into dummy-encoded variables
  • This pandas code:
df = pd.DataFrame(...) # create a pandas DataFrame somehow.
del df['column1'] # delete a column from the dataframe.
df = df.dropna(subset=['column2', 'column3']) # drop rows that have empty values in column 2 and 3.
df = df.rename({'column2': 'unicorns', 'column3': 'dragons'}) # rename column2 and column3
df['newcolumn'] = ['iterable', 'of', 'items'] # add a new column.
- looks like this with pyjanitor:
df = (
pd.DataFrame(...)
.remove_columns(['column1'])
.dropna(subset=['column2', 'column3'])
.rename_column('column2', 'unicorns')
.rename_column('column3', 'dragons')
.add_column('newcolumn', ['iterable', 'of', 'items'])
)

Michael #2: What Does It Take To Be An Expert At Python?

  • Presentation at PyData 2017 by James Powell
  • Covers Python Data Model (dunder methods)
  • Covers uses of Metaclasses
  • All done very smoothly as a series of demos
  • Pretty long and in depth, 1.5+ hours

Brian #3: Awesome Python Applications

  • pypi is a great place to find great packages you can use as examples for the packages you write. Where do you go for application examples? Well, now you can go to Awesome Python Applications.
  • categories of applications included:
    internet, audio, video, graphics, games, productivity, organization, communication, education, science, CMS, ERP (enterprise resource planning), static site generators, and a whole slew of developer related applications.
  • Mahmoud is happy to have help filling this out, so if you know of a great open source application written in Python, go ahead and contribute to this, or open an issue on this project.

Michael #4: Django Core no more

  • Write up by James Bennett
  • If you’re not the sort of person who closely follows the internals of Django’s development, you might not know there’s a draft proposal to drastically change the project’s governance.
  • What’s up: Django the open-source project is OK right now, but difficulty in recruiting and retaining enough active contributors.
  • Some of the biggest open-source projects dodge this by having, effectively, corporate sponsorship of contributions.
  • Django has become sort of a victim of its own success: the types of easy bugfixes and small features that often are the path to growing new committers have mostly been done already in Django.
  • Not managed to bring in new committers at a sufficient rate to replace those who’ve become less active or even entirely inactive, and that’s not sustainable for much longer.
  • Under-attracting women contributors too
  • Governance: Some parallels to what the Python core devs are experiencing now. Project leads BDFLs stepped down.
  • The proposal: what I’ve proposed is the dissolution of “Django core”, and the revocation of almost all commit bits
    • Seems extreme but they were working much more as a team with PRs, etc anyway.
    • Breaks down the barrier to needing to be on the core team to suggest, change anything.
    • Two roles would be formalized — Mergers and Releasers — who would, respectively, merge pull requests into Django, and package/publish releases. But rather than being all-powerful decision-makers, these would be bureaucratic roles

Brian #5: wemake django template

  • a cookie-cutter template for serious django projects with lots of fun goodies
  • “This project is used to scaffold a django project structure. Just like django-admin.py startproject but better.”
  • features:
    • Always up-to-date with the help of [@dependabot](https://dependabot.com/)
    • poetry for managing dependencies
    • mypy for optional static typing
    • pytest for unit testing
    • flake8 and wemake-python-styleguide for linting
    • pre-commit hooks for consistent development
    • docker for development, testing, and production
    • sphinx for documentation
    • Gitlab CI with full build, test, and deploy pipeline configured by default
    • Caddy with https and http/2 turned on by default

Michael #6: Django Hunter

  • Tool designed to help identify incorrectly configured Django applications that are exposing sensitive information.
  • Why? March 2018: 28,165 thousand django servers are exposed on the internet, many are showing secret API keys, database passwords, amazon AWS keys.
  • Example: https://twitter.com/6IX7ine/status/978598496658960384
  • Some complained this inferred Django was insecure and said it wasn’t. Others thought “There is a reasonable argument to be made that DEBUG should default to False.”
  • One beginner, Peter, chimes in:
    • I probably have one of them, among my early projects that are on heroku and public GitHub repos.
    • I did accidentally expose my aws password this way and all hell broke loose.
    • The problem is that as a beginner, it wasn't obvious to me how to separate development and production settings and keep production stuff out of my public repository.

Extras:

Michael: Thanks for having me on your show Brian: https://blog.michaelckennedy.net/2018/12/08/being-a-great-podcast-guest/

Brian: open source extra:
For Christmas, I want a dragon…

pic.twitter.com/RmFAEgqpSr
— Changelog (@changelog)

Michael: Why did the multithreaded chicken cross the road?

  • road the side get to the other of to
  • to get the side to road the of other
  • the side of to the to road other get
  • to of the road to side other the get
Dec 11 2018
21 mins
Play

Rank #13: #85 Visually debugging your Jupyter notebook

Podcast cover
Read more

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean

Brian #1: the state of type hints in Python

  • “Therefore, type hints should be used whenever unit test are worth writing.”
  • Type hints, especially for function arguments and return values, help make your code easier to read, and therefore, easier to maintain.
  • This includes refactoring, allowing IDEs to help with code completion, and allow linters to find problems.
  • For CPython
    • No runtime type inference happens.
    • No performance tuning allowed.
    • Of course, third party packages are not forbidden to do so.
  • Non-comment type annotations are available for functions in 3.0+
  • Variable annotations for 3.6+
  • In 3.7, you can postpone evaluation of annotations with:
    from __future__ import annotations
  • Interface stub files .pyi files, are allowed now, but this is extra work and code to maintain.
    • typeshed has types for standard library plus many popular libraries.
  • How do deal with multiple types, duck typing, and more discussed.
  • A discussion of type generation and checking tools available now, including mypy
  • See also: Stanford Seminar - Optional Static Typing for Python - Talk by Guido van Rossum
    • Interesting discussion that starts with a bit of history of where mypy came from.

Michael #2: Django MongoDB connector

  • Via Robin on Twitter
  • Use MongoDB as the backend for your Django project, without changing the Django ORM.
  • Use Django Admin to access MongoDB
  • Use Django with MongoDB data fields: Use MongoDB embedded documents and embedded arrays in Django Models.
  • Connect 3rd party apps with MongoDB: Apps like Django Rest Framework and Viewflow app that use Django Models integrate easily with MongoDB.
  • Requirements:
    • Python 3.6 or higher.
    • MongoDB 3.4 or higher.
  • Example
inner_qs = Blog.objects.filter(name__contains='Ch').values('name')
entries = Entry.objects.filter(blog__name__in=inner_qs)

Brian #*3: Python Idioms: Multiline Strings*

  • or “How I use dedent”
  • Example:
def create_snippet():
code_snippet = textwrap.dedent("""\
int main(int argc, char* argv[]) {
return 0;
}
""")
do_something(code_snippet)

Michael #4: Flaskerizer

  • A program that automatically creates Flask apps from Bootstrap templates
  • Bootstrap templates from websites like https://Bootstrapmade.com/ and https://startBootstrap.com are a fast way to get very dynamic website up and running
  • Bootstap templates typically don't work "out of the box" with the python web framework Flask and require some tedious directory building and broken link fixing before being functional with Flask.
  • The Flaskerizer automates the necessary directory building and link creation needed to make Bootstrap templates work "out of the box" with Flask.
  • Queue black turtleneck!

Brian #*5: Learn Python the Methodical Way

  • From the article:
    • Make your way through a tutorial/chapter that teaches you some discrete, four-to-six-step skill.
    • Write down those steps as succinctly and generically as possible.
    • Put the tutorial/chapter and its solutions away.
    • Build your project from scratch, peeking only when you’re stuck.
    • Erase what you built.
    • Do the project again.
    • Drink some water.
    • Erase what you built and do it again.
    • A day or two later, delete your work and do it again – this time without peeking even once.
    • Erase your work and do it again.
  • The notion of treating code like you treat creative writing with rough drafts and sometimes complete do-overs is super liberating.
  • You’ll be surprised how fast you can do something the second time, the third time, the fourth time. And it’s very gratifying.

Michael #6: PixieDebugger

  • The Visual Python Debugger for Jupyter Notebooks You’ve Always Wanted
  • Jupyter already supports pdb for simple debugging, where you can manually and sequentially enter commands to do things like inspect variables, set breakpoints, etc.
  • Check out the video to get a good idea of its usage: https://www.youtube.com/watch?v=Z-tPeEkVqjk
Jul 03 2018
24 mins
Play

Rank #14: #95 Unleash the py-spy!

Podcast cover
Read more

Sponsored by DataDog -- pythonbytes.fm/datadog

Brian #1: dataset: databases for lazy people

  • dataset provides a simple abstraction layer removes most direct SQL statements without the necessity for a full ORM model - essentially, databases can be used like a JSON file or NoSQL store.
  • A simple data loading script using dataset might look like this:
import dataset

db = dataset.connect('sqlite:///:memory:')

table = db['sometable']
table.insert(dict(name='John Doe', age=37))
table.insert(dict(name='Jane Doe', age=34, gender='female'))

john = table.find_one(name='John Doe')

Michael #2: CuPy GPU NumPy

  • A NumPy-compatible matrix library accelerated by CUDA
  • How many cores does a modern GPU have?
  • CuPy's interface is highly compatible with NumPy; in most cases it can be used as a drop-in replacement.
  • You can easily make a custom CUDA kernel if you want to make your code run faster, requiring only a small code snippet of C++. CuPy automatically wraps and compiles it to make a CUDA binary
  • PyCon 2018 presentation: Shohei Hido - CuPy: A NumPy-compatible Library for GPU
  • Code example
>>> # This will run on your GPU!
>>> import cupy as np # This is the only non-NumPy line

>>> x = np.arange(6).reshape(2, 3).astype('f')
>>> x
array([[ 0., 1., 2.],
[ 3., 4., 5.]], dtype=float32)
>>> x.sum(axis=1)
array([ 3., 12.], dtype=float32)

Brian #3: Automate Python workflow using pre-commits

  • We covered pre-commit in episode 84, but I still had trouble getting my head around it.
  • This article by LJ Miranda does a great job with the workflow introduction and configuration necessary to get pre-commit working for black and flake8.
  • Includes a nice visual of the flow.
  • Demo of it all in action with a short video.

Michael #4: py-spy

  • Sampling profiler for Python programs
  • Written by Ben Frederickson
  • Lets you visualize what your Python program is spending time on without restarting the program or modifying the code in any way.
  • Written in Rust for speed
  • Doesn't run in the same process as the profiled Python program
  • Does NOT it interrupt the running program in any way.
  • This means Py-Spy is safe to use against production Python code.
  • The default visualization is a top-like live view of your python program
  • How does py-spy work? Py-spy works by directly reading the memory of the python program using the process_vm_readv system call on Linux, the vm_read call on OSX or the ReadProcessMemory call on Windows.

Brian #5: SymPy is a Python library for symbolic mathematics

  • “Symbolic computation deals with the computation of mathematical objects symbolically. This means that the mathematical objects are represented exactly, not approximately, and mathematical expressions with unevaluated variables are left in symbolic form.”
  • example:
>>> integrate(sin(x**2), (x, -oo, oo))
√2⋅√π
─────
2
  • examples on site are interactive so you can play with it without installing anything.

Michael #6: Starlette ASGI web framework

Extras:

Michael: PyCon 2019 dates out, put them on your calendar!

  • Tutorials: May 1-2 • Wednesday, Thursday
  • Talks and Events: May 3–5 • Friday, Saturday, Sunday
  • Sprints: May 6–9 • Monday through Thursday

Listener follow up on git pre-commit hooks util: pre-commit package

  • Matthew Layman, @mblayman
  • Heard the discussion about Git commit hooks at the end. I wanted to bring up pre-commit as an interesting project (written in Python!) that's useful for Git commit hooks.
  • tl;dr:
    • $ pip install pre-commit
    • $ ... create a .pre-commit-config.yaml
    • $ pre-commit install # This is a one time operation.
  • pre-commit's job is to manage a project's Git commit hooks. We use this on my team at work and the devs only need to run pre-commit install. This saves us from a bunch of failing CI builds where flake8 or other code style checks would fail.
  • We use pre-commit to run flake8 and black before allowing a commit to proceed. Some projects have a pre-commit configuration to use right out of the box (e.g., black https://github.com/ambv/black#version-control-integration).

Listener: You don't need that (pattern)

  • John Tocher
  • PyCon AU Talk Called "You don't need that” - by Christopher Neugebauer, it was an interesting take on why with a modern and powerful language like python, you may not need the conventionally described design patterns, ala the "Gang of four".
Sep 15 2018
23 mins
Play

Rank #15: #111 loguru: Python logging made simple

Podcast cover
Read more

Sponsored by https://pythonbytes.fm/datadog

Brian #1: loguru: Python logging made (stupidly) simple

  • Finally, a logging interface that is just slightly more syntax than print to do mostly the right thing, and all that fancy stuff like log rotation is easy to figure out.
  • i.e. a logging API that fits in my brain.
  • bonus: README is a nice tour of features with examples.
  • Features:
    • Ready to use out of the box without boilerplate
    • No Handler, no Formatter, no Filter: one function to rule them all
    • Easier file logging with rotation / retention / compression
    • Modern string formatting using braces style
    • Exceptions catching within threads or main
    • Pretty logging with colors
    • Asynchronous, Thread-safe, Multiprocess-safe
    • Fully descriptive exceptions
    • Structured logging as needed
    • Lazy evaluation of expensive functions
    • Customizable levels
    • Better datetime handling
    • Suitable for scripts and libraries
    • Entirely compatible with standard logging
    • Personalizable defaults through environment variables
    • Convenient parser
    • Exhaustive notifier

Michael #2: Python gets a new governance model

  • by Brett Canon
  • July 2018, Guido steps down
  • Python progress has basically been on hold since then
  • ended up with 7 governance proposals
  • Voting was open to all core developers as we couldn't come up with a reasonable criteria that we all agreed to as to what defined an "active" core dev
  • And the winner is ... In the end PEP 8016, the steering council proposal, won.
  • it was a decisive win against second place
  • PEP 8016 is heavily modeled on the Django project's organization (to the point that the PEP had stuff copy-and-pasted from the original Django governance proposal).
    • What it establishes is a steering council of five people who are to determine how to run the Python project. Short of not being able to influence how the council itself is elected (which includes how the electorate is selected), the council has absolute power.
    • result of the vote prevents us from ever having the Python project be leaderless again, it doesn't directly solve how to guide the language's design.
  • What's next? The next step is we elect the council. It's looking like nominations will be from Monday, January 07 to Sunday, January 20 and voting from Monday, January 21 to Sunday, February 03
  • A key point I hope people understand is that while we solved the issue of project management that stemmed from Guido's retirement, the council will need to be given some time to solve the other issue of how to manage the design of Python itself.

Brian #3: Why you should be using pathlib

  • Tour of pathlib from Trey Hunner
  • pathlib combines most of the commonly used file and directory operations from os, os.path, and glob.
  • uses objects instead of strings
  • as of Python 3.6, many parts of stdlib support pathlib
  • since pathlib.Path methods return Path objects, chaining is possible
  • convert back to strings if you really need to for pre-3.6 code
  • Examples:
    • make a directory: Path('src/__pypackages__').mkdir(parents=True, exist_ok=True)
    • rename a file: Path('.editorconfig').rename('src/.editorconfig')
    • find some files: top_level_csv_files = Path.cwd().glob('*.csv')
    • recursively: all_csv_files = Path.cwd().rglob('*.csv')
    • read a file: Path('some/file').read_text()
    • write to a file: Path('.editorconfig').write_text('# config goes here')
    • with open(path, mode) as x works with Path objects as of 3.6
  • Follow up article by Trey: No really, pathlib is great

Michael #4: Altair and Altair Recipes

  • via Antonio Piccolboni (he wrote altair_recipes)
  • Altair: Declarative statistical visualization library for Python
    • Altair is developed by Jake Vanderplas and Brian Granger
    • By statistical visualization they mean:
      • The data source is a DataFrame that consists of columns of different data types (quantitative, ordinal, nominal and date/time).
      • The DataFrame is in a tidy format where the rows correspond to samples and the columns correspond to the observed variables.
      • The data is mapped to the visual properties (position, color, size, shape, faceting, etc.) using the group-by data transformation.
    • Nice example that I can get behind
# cars = some Pandas data frame
alt.Chart(cars).mark_point().encode(
x='Horsepower',
y='Miles_per_Gallon',
color='Origin',
)
  • altair_recipes
    • Altair allows generating a wide variety of statistical graphics in a concise language, but lacks, by design, pre-cooked and ready to eat statistical graphics, like the boxplot or the histogram.
    • Examples: https://altair-recipes.readthedocs.io/en/latest/examples.html
    • They take a few lines only in altair, but I think they deserve to be one-liners. altair_recipes provides that level on top of altair. The idea is not to provide a multitude of creative plots with fantasy names (the way seaborn does) but a solid collection of classics that everyone understands and cover most major use cases: the scatter plot, the boxplot, the histogram etc.
    • Fully documented, highly consistent API (see next package), 90%+ test coverage, maintainability grade A, this is professional stuff if I may say so myself.

Brian #5: A couple fun pytest plugins

  • pytest-picked
    • Using git status, this plugin allows you to:
      • Run only tests from modified test files
      • Run tests from modified test files first, followed by all unmodified tests
    • Kinda hard to overstate the usefulness of this plugin to anyone developing or debugging a test. Very, very cool.
  • pytest-clarity
    • Colorized left/right comparisons
    • Early in development, but already helpful.
    • I recommend running it with -qq if you don’t normally run with -v/--verbose since it overrides the verbosity currently.

Michael #6: Secure 🔒 headers and cookies for Python web frameworks

  • Python package called Secure, which sets security headers and cookies (as a start) for Python web frameworks.
  • I was listening to the Talk Python To Me episode “Flask goes 1.0” with Flask maintainer David Lord. At the end of the interview he was asked about notable PyPI packages and spoke about Flask-Talisman, a third-party package to set security headers in Flask. As a security professional, it was surprising and encouraging to hear the maintainer of the most popular Python web framework speak passionately about a security package.
  • Had been recently experimenting with emerging Python web frameworks and realized there was a gap in security packages. That inspired Caleb to (humbly) see if it were possible to make a package to correct that and I started with Responder and then expanded to support more frameworks.
  • The outcome was Secure with functions to support aiohttp, Bottle, CherryPy, Falcon, hug, Pyramid, Quart, Responder, Sanic, Starlette and Tornado (most of these, if not all have been featured on Talk Python) and can also be utilized by frameworks not officially supported. The goal is to be minimalistic, lightweight and be implemented in a way that does not disrupt an individual framework’s design.
  • I have had some great feedback and suggestions from the developer and OWASP community, including some awesome discussions with the OWASP Secure Project and the Sanic core team.
  • Added support for Flask and Django too.
  • Secure Cookies is nice in the mix

Extras:

Michael: SQLite bug impacts thousands of apps, including all Chromium-based browsers

Michael: Follow up to our AI and healthcare conversation

  • via Bradley Hintze
  • I found your discussion of deep learning in healthcare interesting, no doubt because that is my area. I am the data scientist for the National Oncology Program at the Veterans Health Administration.
  • I work directly with clinicians and it is my strong opinion that AI cannot take the job from the MD. It will however make caring for patients much more efficient as AI takes care of the low hanging fruit, it you will.
  • Healthcare, believe it or not, is a science and an art. This is why AI is never going to make doctors obsolete. It will, however, make doctors more efficient and demanded a more sophisticated doctor -- one that understands AI enough to not only trust it but, crucially, comprehend its limits.

Michael: Upgrade to Python 3.7.2

  • If you install via home brew, it’s time for brew update && brew upgrade

Michael: New course!

Jan 05 2019
34 mins
Play

Rank #16: #131 Python 3 has issues (over on GitHub)

Podcast cover
Read more

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean

Brian #1: PEP 581 (Using GitHub issues for CPython) is accepted

  • PEP 581
  • The email announcing the acceptance.
  • “The migration will be a large effort, with much planning, development, and testing, and we welcome volunteers who wish to help make it a reality. I look forward to your contributions on PEP 588 and the actual work of migrating issues to GitHub.” — Barry Warsaw

Michael #2: Replace Nested Conditional with Guard Clauses

# BAD!
def checkout(user):
shipping, express = [], []
if user is not None:
for item in user.cart:
if item.is_available:
shipping.append(item)
if item.express_selected:
express.append(item)

return shipping, express
# BETTER!
def checkout(user):
shipping, express = [], []
if user is None:
return shipping, express

for item in user.cart:
if not item.is_available:
continue

shipping.append(item)
if item.express_selected:
express.append(item)

return shipping, express

Brian #3: Things you’re probably not using in Python 3 – but should

  • Vinko Kodžoman
  • Some of course items:
    • f-strings
    • Pathlib (side note. pytest tmp_path fixture creates temporary directories and files with PathLib)
    • data classes
  • Some I’m warming to:
    • type hinting
  • And those I’m really glad for the reminder of:
  • enumerations
from enum import Enum, auto
class Monster(Enum):
ZOMBIE = auto()
WARRIOR = auto()
BEAR = auto()

print(Monster.ZOMBIE)
# Monster.ZOMBIE
  • built in lru_cache: easy memoization with the functools.lru_cache decorator.
@lru_cache(maxsize=512)
def fib_memoization(number: int) -> int:
...
  • extended iterable unpacking
>>> head, *body, tail = range(5)
>>> print(head, body, tail)
0 [1, 2, 3] 4
>>> py, filename, *cmds = "python3.7 script.py -n 5 -l 15".split()
>>> cmds
['-n', '5', '-l', '15']
>>> first, _, third, *_ = range(10)
>>> first, third
(0, 2)

Michael #4: The Python Arcade Library

  • Arcade is an easy-to-learn Python library for creating 2D video games. It is ideal for people learning to program, or developers that want to code a 2D game without learning a complex framework.
  • Minesweeper games, hangman, platformer games in general.
  • Check out Sample Games Made With The Arcade Library too
  • Includes physics and other goodies
  • Based on OpenGL

Brian #5: Teaching a kid to code with Pygame Zero

  • Matt Layman
  • Scratch too far removed from coding.
  • Using Mu to simplify coding interface.
    • comes with a built in Python.
    • Pygame Zero preinstalled
  • [Pygame Zero] is intended for use in education, so that teachers can teach basic programming without needing to explain the Pygame API or write an event loop.”
  • Initial 29 line game taught:
    • naming things and variables
    • mutability and fiddling with “constants” to see the effect
    • functions and side effects
    • state and time
    • interactions and mouse events
  • Article also includes some tips on how to behave as the adult when working with kids and coding.

Michael #6: Follow up on GIL / PEP 554

  • Has the Python GIL been slain? by Anthony Shaw
  • multithreading in CPython is easy, but it’s not truly concurrent, and multiprocessing is concurrent but has a significant overhead.
  • Because Interpreter state contains the memory allocation arena, a collection of all pointers to Python objects (local and global), sub-interpreters in PEP 554 cannot access the global variables of other interpreters.
  • the way to share objects between interpreters would be to serialize them and use a form of IPC (network, disk or shared memory). All options are fairly inefficient
  • But: PEP 574 proposes a new pickle protocol (v5) which has support for allowing memory buffers to be handled separately from the rest of the pickle stream.
  • When? Pickle v5 and shared memory for multiprocessing will likely be Python 3.8 (October 2019) and sub-interpreters will be between 3.8 and 3.9.

Extras

Brian:

Michael:

Jokes

  • MK → Waiter: Would you like coffee or tea? Programmer: Yes.
May 21 2019
27 mins
Play

Rank #17: #142 There's a bandit in the Python space

Podcast cover
Read more

Special guest: Brett Thomas

Sponsored by Datadog: pythonbytes.fm/datadog

Brian #1: Writing sustainable Python scripts

  • Vincent Bernat
  • Turning a quick Python script into a maintainable bit of software.
  • Topics covered:
    • Documentation as a docstring helps future users/maintainers know what problem you are solving.
    • CLI arguments with defaults instead of hardcoded values help extend the usability of the script.
    • Logging. Including debug logging (and how to turn them on with CLI arguments), and system logging for unattended scripts.
    • Tests. Simple doctests, and pytest tests utilizing parametrize to have one test and many test cases.

Brett #2: Static Analysis and Bandit

Michael #3: jupyter-black

  • Black formatter for Jupyter Notebook
  • One of the big gripes I have about these online editors is their formatting (often entirely absent)
  • Then the extension provides
    • a toolbar button
    • a keyboard shortcut for reformatting the current code-cell (default: Ctrl-B)
    • a keyboard shortcut for reformatting whole code-cells (default: Ctrl-Shift-B)

Brian #4: Report Generation workflow with papermill, jupyter, rclone, nbconvert, …

Brett #5: Rant on time deltas

Michael #6: How — and why — you should use Python Generators

  • by Radu Raicea
  • Generator functions allow you to declare a function that behaves like an iterator.
  • They allow programmers to make an iterator in a fast, easy, and clean way.
  • They only compute it when you ask for it. This is known as lazy evaluation.
  • If you’re not using generators, you’re missing a powerful feature
  • Often they result in simpler code than with lists and standard functions

Extras

Brian:

Michael:

Jokes

A good programmer is someone who always looks both ways before crossing a one-way street.

(reminds me of another joke: Adulthood is like looking both ways before crossing the street, then getting hit by an airplane)

Little bobby tables

Aug 06 2019
30 mins
Play

Rank #18: #65 Speed of your import statements affecting performance?

Podcast cover
Read more

Sponsored by Rollbar: pythonbytes.fm/rollbar

Brian #1: pygal : Simple Python Charting

  • Output SVG or PNG
  • Example Flask App (also django response) part of documentation.
  • Enough other bits of doc to get you a chart in a web page super fast.

Michael #2: Thoughts on becoming a self-taught programming

  • Basic format:
  • I'm 31 days into self-studying Python and am loving every minute of it!
  • A few questions:
    • What were you doing before you began self-studying programming?
    • What made you want to study programming on your own?
    • How did you start (which resources and language)?
    • How long did it take for you to feel confident enough in your skills and knowledge to know you could be employed as a programmer?
    • What else did you do besides self-study that helped you in your journey to becoming a programmer?
    • What's next for you?

Brian #3: How to speed up Python application startup time (timing imports in 3.7)

  • Python 3.7 includes -X importtime option that allows you to profile the time it takes to do all the imports.
  • Way cool tool to help optimize the startup time of an application.

Michael #4: AnPyLar - The Python web front-end framework

  • Create web applications with elegance, simplicity and yet full power with Python and components
  • MISSION: Empower all Python programmers to work not only on the back-end but also on the front-end with the same language of choice
  • Features
    • Reactive programming and Promises
    • Python standard formatting as templates
    • reusable components
    • Scoped styling for component
    • Integrated routing engine

Brian #5: Migrating to Python 3 with pleasure

  • “A short guide on features of Python 3 for data scientists”
  • Quick tutorial through examples of pathlib.
  • Type hinting and how cool it works with editors (PyCharm example shown)
  • Adding runtime type enforcement for specific methods using enforce
  • Using function annotations for units, as done in astropy.
  • Matrix multiplication with @.
  • Globbing with **.
    • found_images = glob.glob('/path/**/*.jpg', recursive=True)
  • Also … underscores in numeric literals, f-strings, true division with /, integer division with //, and lots of more fun goodies.

Michael #6: Moving to Python 3

  • Many of these issues were corrected just by running 2to3, which not only fixed many of the compatibility issues
    • Outdated external libraries which needed to be updated to newer versions featuring Python 3 compatibility
      basestring to str, urlparse to urllib.urlparse and similar major changes
    • Dictionary change like iteritems() to items(), or .items() now returning a view.
    • Things that weren't needed anymore, like Django's force_unicode or __future__ library tools.
  • Once we finished working on the "low-hanging fruits", the next step was to run Aphrodite's test suite and achieve zero errors.
  • Lessons learned
    • Code coverage was originally around 70%,
    • Keeping the Python 3 branch up to date with master
    • A non-trivial feature was delivered during the migration (via feature branch)
    • The pickle protocol version in python 3 can be higher than the highest available in Python 2.7. So we needed to add versioning to our Django caches
    • Each modified file had to comply with flake8 linting rules
  • Afrodita is currently running on Google App Engine Flexible, and one of the features our team loves with is traffic splitting
  • With this feature, we can do canary releases with ease: We just deploy our new version of the service, and start redirecting small amounts of traffic traffic while we monitor for unexpected errors.
  • After some minor bugfixes, we could bring the traffic of the Python 3.6 version to 100% with confidence. We also had the old version available for instant rollback, thanks to how parallel versions and traffic splitting work in GAE flexible.

Our news

Brian:

Michael:

Feb 14 2018
27 mins
Play

Rank #19: #120 AWS, MongoDB, and the Economic Realities of Open Source and more

Podcast cover
Read more

Sponsored by pythonbytes.fm/digitalocean

Brian #1: The Ultimate Guide To Memorable Tech Talks

  • Nina Zakharenko
  • 7 part series that covers choosing a topic, writing a talk proposal, tools, planning, writing, practicing, and delivering the talk
  • I’ve just read the tools section, and am looking forward to the rest of the series.
    • From the tools section: “I noticed I’d procrastinate on making the slides look good instead of focusing my time on making quality content.”
Michael #2: Running Flask on Kubernetes

  • via TestDriven.io & Michael Herman
  • What is Kubernetes?
  • A step-by-step tutorial that details how to deploy a Flask-based microservice (along with Postgres and Vue.js) to a Kubernetes cluster.
  • Goals of tutorial
    1. Explain what container orchestration is and why you may need to use an orchestration tool
    2. Discuss the pros and cons of using Kubernetes over other orchestration tools like Docker Swarm and Elastic Container Service (ECS)
    3. Explain the following Kubernetes primitives - Node, Pod, Service, Label, Deployment, Ingress, and Volume
    4. Spin up a Python-based microservice locally with Docker Compose
    5. Configure a Kubernetes cluster to run locally with Minikube
    6. Set up a volume to hold Postgres data within a Kubernetes cluster
    7. Use Kubernetes Secrets to manage sensitive information
    8. Run Flask, Gunicorn, Postgres, and Vue on Kubernetes
    9. Expose Flask and Vue to external users via an Ingress
Brian #3: Changes in the CI landscape

Michael #4: Python server setup for macOS 🍎

  • what: hello world for Python server setup on macOS
  • why: most guides show setup on a Linux server (which makes sense) but macoS is useful for learning and for local dev
  • STEP 1: NGINX ➡️ STATIC ASSETS
  • STEP 2: GUNICORN ➡️ FLASK
  • STEP 3: NGINX ➡️ GUNICORN
Brian #5: Learn Enough Python to be Useful: argparse

  • How to Get Command Line Arguments Into Your Scripts - Jeff Hale
  • “argparse is the “recommended command-line parsing module in the Python standard library.” It’s what you use to get command line arguments into your program.
  • “I couldn’t find a good intro guide for argparse when I needed one, so I wrote this article.”
Michael #6: AWS, MongoDB, and the Economic Realities of Open Source

  • Related podcast: https://soundcloud.com/exponentfm/episode-159-inverted-pyramids
  • Last week, from the AWS blog:

    Today we are launching Amazon DocumentDB (with MongoDB compatibility), a fast, scalable, and highly available document database that is designed to be compatible with your existing MongoDB applications and tools. Amazon DocumentDB uses a purpose-built SSD-based storage layer, with 6x replication across 3 separate Availability Zones. The storage layer is distributed, fault-tolerant, and self-healing, giving you the the performance, scalability, and availability needed to run production-scale MongoDB workloads.

  • Like an increasing number of such projects, MongoDB is open source…or it was anyways. MongoDB Inc., a venture-backed company that IPO’d in October, 2017, made its core database server product available under the GNU Affero General Public License (AGPL).

  • AGPL extended the GPL to apply to software accessed over a network; since the software is only being used, not copied
  • MongoDB’s Business Model
  • We believe we have a highly differentiated business model that combines the developer mindshare and adoption benefits of open source with the economic benefits of a proprietary software subscription business model.
    • MongoDB enterprise and MongoDB atlas
  • Basically, MongoDB sells three things on top of its open source database server:
    • Additional tools for enterprise companies to implement MongoDB
    • A hosted service for smaller companies to use MongoDB
    • Legal certainty
  • What AWS Sells
  • the value of software is typically realized in three ways:
    • First is hardware.
    • Second is licenses. This was Microsoft’s core business for decades: licenses sold to OEMs (for the consumer market) or to companies directly (for the enterprise market).
    • Third is software-as-a-service.
  • AWS announced last week:
    > The storage layer is distributed, fault-tolerant, and self-healing, giving you the the performance, scalability, and availability needed to run production-scale MongoDB workloads.
  • AWS is not selling MongoDB: what they are selling is “performance, scalability, and availability.” DocumentDB is just one particular area of many where those benefits are manifested on AWS.
  • Thus we have arrived at a conundrum for open source companies:
    • MongoDB leveraged open source to gain mindshare.
    • MongoDB Inc. built a successful company selling additional tools for enterprises to run MongoDB.
    • More and more enterprises don’t want to run their own software: they want to hire AWS (or Microsoft or Google) to run it for them, because they value performance, scalability, and availability.
  • This leaves MongoDB Inc. not unlike the record companies after the advent of downloads: what they sold was not software but rather the tools that made that software usable, but those tools are increasingly obsolete as computing moves to the cloud. And now AWS is selling what enterprises really want.
  • This tradeoff is inescapable, and it is fair to wonder if the golden age of VC-funded open source companies will start to fade (although not open source generally). The monetization model depends on the friction of on-premise software; once cloud computing is dominant, the economic model is much more challenging.
Extras:

PyTexas 2019 at #Austin on Apr 13th and 14th. Registrations now open. More info at pytexas.org/2019/

Michael: Sorry Ant!

Michael: RustPython follow up: https://rustpython.github.io/demo/

Joke:

  • Q: Why was the developer unhappy at their job?
  • A: They wanted arrays.

  • Q: Where did the parallel function wash its hands?

  • A: Async
Mar 05 2019
25 mins
Play

Rank #20: #117 Is this the end of Python virtual environments?

Podcast cover
Read more

Sponsored by pythonbytes.fm/datadog

Brian #1: Goodbye Virtual Environments?

  • by Chad Smith
  • venv’s are great but they introduce some problems as well:
    • Learning curve: explaining “virtual environments” to people who just want to jump in and code is not always easy
    • Terminal isolation: Virtual Environments are activated and deactivated on a per-terminal basis
    • Cognitive overhead: Setting up, remembering installation location, activating/deactivating
  • PEP 582 — Python local packages directory
    • This PEP proposes to add to Python a mechanism to automatically recognize a __pypackages__directory and prefer importing packages installed in this location over user or global site-packages. This will avoid the steps to create, activate or deactivate “virtual environments”. Python will use the __pypackages__ from the base directory of the script when present.
  • Try it now with pythonloc
    • pythonloc is a drop in replacement for python and pip that automatically recognizes a __pypackages__ directory and prefers importing packages installed in this location over user or global site-packages. If you are familiar with node, __pypackages__ works similarly to node_modules.
    • Instead of running python you run pythonloc and the __pypackages__ path will automatically be searched first for packages. And instead of running pip you run piploc and it will install/uninstall from __pypackages__.

Michael #2: webassets

  • Bundles and minifies CSS & JS files
  • Been doing a lot of work to rank higher on the sites
  • That lead me to Google’s Lighthouse
  • Despite 25ms response time to the network, Google thought my site was “kinda slow”, yikes!
  • webassets has integration for the big three: Django, Flask, & Pyramid.
    • But I prefer to just generate them and serve them off disk
def build_asset(env: webassets.Environment,
files: List[str],
filters: str,
output: str):
bundle = webassets.Bundle(
*files,
filters=filters,
output=output,
env=env
)
bundle.build(force=True)

Brian #3: Bernat on Python Packaging

Michael #4: What the mock? — A cheatsheet for mocking in Python

  • Nice introduction
  • Some examples
@mock.patch('work.os')
def test_using_decorator(self, mocked_os):
work_on()
mocked_os.getcwd.assert_called_once()

And

def test_using_context_manager(self):
with mock.patch('work.os') as mocked_os:
work_on()
mocked_os.getcwd.assert_called_once()

Brian #5: Transitions: The easiest way to improve your tech talk

  • By Saron Yitbarek
  • Jeff Atwood of CodingHorror noted “The people who can write and communicate effectively are, all too often, the only people who get heard. They get to set the terms of the debate.”
  • Effectively presenting is part of effective communication.
  • I love the focus of this article. Focused on one little aspect of improving the performance of a tech talk.

Michael #6: Steering council announced

  • Our new leaders are
    • Barry Warsaw
    • Brett Cannon
    • Carol Willing
    • Guido van Rossum
    • Nick Coghlan
  • Via Joe Carey
  • We both think it’s great Guido is on the council.

From the list from Ant, my votes.

  • Q: What's the second movie about a database engineer called?
    A: The SQL.

  • !false
    It's funny 'cause it's true.

  • A programmer's spouse tells them, "Run to the store and pick up a loaf of bread. If they have eggs, get a dozen."
    The programmer comes home with 12 loaves of bread.

Feb 14 2019
28 mins
Play

Similar Podcasts