* apologies to Eugene Wigner
We all know that computers are great at doing repetitive tasks, but often fail to realize that there are lots of ways we can use computers to can make our own daily work more efficient. We can be a lot more effective about how we use our computers if we use them as a tool for ourselves.
The shell is the primary way you interact with a computer once you want to do more than what visual user interfaces allow. Visual user interfaces are somewhat limited in what they can allow because you can only do the things that there are buttons and sliders and input fields for.
Textual tools are often built to be composable: providing many different ways to combine, program or automate them. Using the computer as a tool for ourselves starts with understanding a few basic things: how to use your shell, terminal, and a programming editor.
Many software developers are captives of button-pushing repetitive drudgery. The cobbler's children have no shoes. As they are developing tools for others, they fail to use the computer effectively as a tool for themselves.
Past examples from my own observation:
- tests are manual, results are word docs of pasted screenshots
- code is written in Word or notepad
- code is kept in a scattering of "secret" places, only a handful of people have access to, or not kept at all
- documentation is separate from code, all hand-written, inaccessible/obfuscated in proprietary systems, or just missing
- deployment is via runbook Word doc: a detailed step-by-step instructions for humans to execute
A wishlist for using the computer as a tool for myself: automating nearly all of the repetitive drudgery that goes into software development, leaving more time for the fun parts:
- streamlined management of all your source, docs and related artifacts in a distributed fashion to track changes, revision history, rollback breaking changes, collaborate with team, isolate changes from main codebase, troubleshoot changes, and much more...
- all aspects of which are managed across multiple versions of software
- within a framework for project packaging and package dependencies
- and a framework for unit testing, automated across all versions
- and automated code coverage analysis
- automated testing, continuous integration and continuous delivery across multiple environments
- automated lint checks for code style, likely bugs, design flaws, import order
- documentation of code in-place, automated generation of docs, automated checking for doc standards conformance
- automated check of dependencies for known security vulnerabilities
- automated decoration of untyped code with type annotations, then used for static and runtime type checking
- auto-formatting of code on pre-commit
- automatic test that in-line docs match code and docstring examples match documented example output
- status tracking of continuous integration tests and doc tests with dashboard badges
- automatically create release notes
- automatically publish to a package repository for easy deployments/updates
- automatically publish versioned docs to a document repository, auto-rebuilt on project update
Most shops use computers as tools to automate a small fraction of this list. Most of this list is just skipped and not done. The essential bits are handled with repetitive human drudgery.
I've recently been playing with Python frameworks that deliver every single bit of the ENTIRE wishlist above and it has provided much food for thought. I would love to apply the same to an end-to-end data engineering solution. Next step, become familiar with DBT!
get_wp project on GitHub: https://github.com/SteveRutledge/get_wp
No comments:
Post a Comment