比如说，Luigi就是解决问题的第三次尝试。前两个尝试解决了错误的问题，并且为错误的方向做了优化。比如第一个版本依赖于在 XML 中设计依赖图。但是很显然这是非常不友好的，因为你一般县要在代码里生成依赖图比较好。而且，在前两次设计中看起来很有用的一些新设计，比如任务解耦输出，最终只给一些非常少见的例子添加了支持，但是有添加了不少复杂度。
第一个版本中看起来很奇怪的问题可能在后来是很重要的问题，反过来也是。
I was reminded of this when we built an email ingestion system at Better. The first attempt failed because we built it in a poor way (basically shoehorning it into a CRUD request). The second one had a solid microservice design but failed for usability reasons (we built a product that no one really asked for). We’re halfway through the third attempt and I’m having a good feeling about it.
There are 3 kind of queues: `Queue LifoQueue HeapQueue
q = Queue(size)
get(block=True) return a object
get_nowait()
put(item, block=True) put a object
qsize()
empty()
full()
task_done() # indicate one item process finished raise ValueError
join() # wait until all item processed
Queue.Empty
Queue.Full
How to iterate a queue?
by using iter(q, None), note that you have to put a sentinel value manually
Queue vs multiprocessing.Queue
despite their similar api, their implementation is completely different
A file modu.py in the directory pack/ is imported with the statement import pack.modu. This statement will look for an __init__.py file in pack, execute all of its top-level statements. Then it will look for a file named pack/modu.py and execute all of its top-level statements. After these operations, any variable, function, or class defined in modu.py is available in the pack.modu namespace.
A commonly seen issue is to add too much code to __init__.py files. When the project complexity grows, there may be sub-packages and sub-sub-packages in a deep directory structure. In this case, importing a single item from a sub-sub-package will require executing all __init__.py files met while traversing the tree.
This and other issues led to the idea that using stateless functions is a better programming paradigm.
Another way to say the same thing is to suggest using functions and procedures with as few implicit contexts and side-effects as possible. A function’s implicit context is made up of any of the global variables or items in the persistence layer that are accessed from within the function. Side-effects are the changes that a function makes to its implicit context. If a function saves or deletes data in a global variable or in the persistence layer, it is said to have a side-effect.
Pure functions
Pure functions are deterministic: given a fixed input, the output will always be the same.
Pure functions are much easier to change or replace if they need to be refactored or optimized.
Pure functions are easier to test with unit-tests: There is less need for complex context setup and data cleaning afterwards.
Pure functions are easier to manipulate, decorate, and pass around.
However, it may be a good discipline to avoid assigning to a variable more than once, and it helps in grasping the concept of mutable and immutable types.
A LICENSE file should always be present and specify the license under which the software is made available to the public.
A TODO file or a TODO section in README should list the planned development for the code.
A CHANGELOG file or section in README should compile a short overview of the changes in the code base for the latest versions.
Project Publication
Depending on the project, your documentation might include some or all of the following components:
An introduction should show a very short overview of what can be done with the product, using one or two extremely simplified use cases. This is the thirty-second pitch for your project.
A tutorial should show some primary use cases in more detail. The reader will follow a step-by-step procedure to set-up a working prototype.
An API reference is typically generated from the code (see docstrings). It will list all publicly available interfaces, parameters, and return values.
Developer documentation is intended for potential contributors. This can include code convention and general design strategy of the project.
put project/ tests/ docs/ directory in side the project
Decorator to wrap a function with a memoizing callable that saves up to the maxsize most recent calls. It can save time when an expensive or I/O bound function is periodically called with the same arguments.
Since a dictionary is used to cache results, the positional and keyword arguments to the function must be hashable.
If maxsize is set to None, the LRU feature is disabled and the cache can grow without bound. The LRU feature performs best when maxsize is a power-of-two.
If typed is set to true, function arguments of different types will be cached separately. For example, f(3) and f(3.0) will be treated as distinct calls with distinct results.
To help measure the effectiveness of the cache and tune the maxsize parameter, the wrapped function is instrumented with a cacheinfo() function that returns a named tuple showing hits, misses, maxsize and currsize. In a multi-threaded environment, the hits and misses are approximate.
The decorator also provides a cacheclear() function for clearing or invalidating the cache.
singledispatch
To define a generic function, decorate it with the @singledispatch decorator. Note that the dispatch happens on the type of the first argument, create your function accordingly
