#180 Transactional file IO with Python and safer
Published May 8, 2020
32 min
    Add to queue
    Copy URL
    Show notes

    Sponsored by DigitalOcean: pythonbytes.fm/digitalocean - $100 credit for new users to build something awesome.

    Michael #1: Ubuntu 20.04 is out!

    • Next LTS support version since 26th April 2018 (18.04).
    • Comes with Python 3.8 included!
    • Already upgraded all our servers, super smooth.
    • Kernel has been updated to the 5.4 based Linux kernel, with additional support for Wireguard VPN, AUFS5, and improved support for IBM, Intel, Raspberry Pi and AMD hardware.
    • Features the latest version of the GNOME desktop environment.
    • Brings support for installing an Ubuntu desktop system on top of ZFS.
    • 20.04 already an option on DigitalOcean ;)

    Brian #2: Working with warnings in Python

    • (Or: When is an exception not an exception?)
    • Reuven Lerner
    • Exceptions, the class hierarchy of exceptions, and warnings.
    • “… most of the time, warnings are aimed at developers rather than users. Warnings in Python are sort of like the “service needed” light on a car; the user might know that something is wrong, but only a qualified repairperson will know what to do. Developers should avoid showing warnings to end users.”
    • Python’s warning system …:
      • It treats the warnings as a separate type of output, so that we cannot confuse it with either exceptions or the program’s printed text,
      • It lets us indicate what kind of warning we’re sending the user,
      • It lets the user indicate what should happen with different types of warnings, with some causing fatal errors, others displaying their messages on the screen, and still others being ignored,
      • It lets programmers develop their own, new kinds of warnings.
    • Reuven goes on to show how to use warnings in your code.
      • using them
      • creating custom warnings
      • filtering

    Michael #3: Safer file writer

        with open(filename, 'w') as fp:
            json.dump(data, fp)
    • It’s using with, so it’s good right?
    • Well the file itself may be overwritten and maybe corrupted
    • With safer, you write almost identical code:
    with safer.open(filename, 'w') as fp:
        json.dump(data, fp)

    Brian #4: codespell

    • codespell : Fix common misspellings in text files. It's designed primarily for checking misspelled words in source code, but it can be used with other files as well.
    • I got a cool pull request against the cards project to add a pre-commit hook to run codespell. (Thanks Christian Clauss)
    • codespell caught a documentation spelling error in cards, where I had spelled “arguments” as “arguements”. Oops.
    • Spelling errors are annoying and embarrassing in code and comments, and distracting. Also hard to deal with using traditional spell checkers. So super glad this is a thing.

    Michael #5: Austin profiler

    • via Anthony Shaw
    • Python frame stack sampler for CPython
    • Profiles CPU and Memory!
    • Why Austin?
      • Written in pure C Austin is written in pure C code. There are no dependencies on third-party libraries.
      • Just a sampler - fast: Austin is just a frame stack sampler. It looks into a running Python application at regular intervals of time and dumps whatever frame stack it finds.
      • Simple output, powerful tools Austin uses the collapsed stack format of FlameGraph that is easy to parse. You can then go and build your own tool to analyse Austin's output.
      • You could even make a player that replays the application execution in slow motion, so that you can see what has happened in temporal order.
      • Small size Austin compiles to a single binary executable of just a bunch of KB.
      • Easy to maintain Occasionally, the Python C API changes and Austin will need to be adjusted to new releases. However, given that Austin, like CPython, is written in C, implementing the new changes is rather straight-forward.
    • Creates nice flame graphs
    • The Austin TUI is nice! Austin TUI

    • Web Austin is yet another example of how to use Austin to make a profiling tool. It makes use of d3-flame-graph to display a live flame graph in the web browser that refreshes every 3 seconds with newly collected samples.

    • Austin output format can be converted easily into the Speedscope JSON format. You can find a sample utility along with the TUI and Austin Web.

    Brian #6: Numbers in Python

    • Moshe Zadka
    • A great article on integers, floats, fractions, & decimals
    • Integers
      • They turn into floats very easily, (4/3)*34.0, int → float
    • Floats
      • don’t behave like the floating point numbers in theory
      • don’t obey mathematical properties
        • subtraction and addition are not inverses
          • 0.1 + 0.2 - 0.2 - 0.1 != 0.0
        • addition is not associative
      • My added comment: Don’t compare floats with ==, use pytest.approx or other approximation techniques.
    • Fractions
      • Kinda cool that they are there but be very careful about your input
      • Algorithms on fractions can explode in time and to some extent memory.
      • Generally better to use floats
    • Decimals
      • Good for financial transactions.
      • Weird dependence on a global state variable, the context precision.
      • Safer to use a local context to set the precision locally
        >>> with localcontext() as ctx:
        ...     ctx.prec = 10
        ...     Decimal(1) / Decimal(7)





    Unix is user friendly. It's just very particular about who its friends are. (via PyJoke)

    If you put 1000 monkeys at 1000 computers eventually one will write a Python program. The rest will write PERL. (via @JamesAbel)

        0:00:00 / 0:00:00