Python

I have been reading through some of the source code from the requests library. This is one of the more well-known, well-used, well-loved, and well-praised Python libraries I know of. I came across some conditional code which I personally would refactor. A first refactor I’m pretty confident about whilst a second I’m less certain is an improvement. I’ll detail them both and welcome any comments anyone has. A specific example of this kind of conditional comes in the adapters module specifically the cert_verify method. Here is the code in question, slightly snipped for brevity: ...

Developing a small web application I recently had reason to upgrade from Python 3.4 to Python 3.6. The reason for the upgrade was regarding ordering of keyword arguments and not related to the bug in my test code that I then found. I should have been more careful writing my test code in the first place, so I am writing this down as some penance for not testing my tests robustly enough. ...

I have written before about Flask and obtaining test coverage results here and with an update here. This is pretty trivial if you’re writing unit tests that directly call the application, but if you actually want to write tests which animate a browser, for example with selenium, then it’s a little more complicated, because the browser/test code has to run concurrently with the server code. Previously I would have the Flask server run in a separate process and run ‘coverage’ over that process. This was slightly unsatisfying, partly because you sometimes want coverage analysis of your actual tests. Test suites, just like application code, can grow in size with many utility functions and imports etc. which may eventually end up not actually being used. So it is good to know that you’re not needlessly maintaining some test code which is not actually invoked. ...

I come from a functional programming background, so I a lot of love for functions and so-called anonymous functions or lambdas. However, I have realised that I don’t make use of Python’s lambda syntax much, and I wanted to articulate why. This may mostly sound pretty negative towards lambdas, but bear in mind I’m certainly not against lambdas in general. A lambda expression could always be avoided by simply defining the function in question. So the question becomes when is a lambda more readable than a definition? That is almost always because the function you’re attempting to create is so simple that the definition syntax gets in the way. It is often used when the function in question is going to be used as an argument to some method. For example the sort method on a list takes as argument they key to be used when comparing the elements. So if you have a list of items and you want to sort them by their prices you can do something such as: ...

Looking through the code of an open-source Python project I came across code that amounts to the following: host_name = self.connection.getsockname()[0] if self.server.host_name is not None: host_name = self.server.host_name This looks odd because if the condition is True then we forget about the host_name derived from the call to self.connection.getsockname. However it could be that that call has some side-effect. Assuming that it does not we can re-write the above code as: if self.server.host_name is not None: host_name = self.server.host_name else: host_name = self.connection.getsockname()[0] Or even, if you prefer: ...

More than a decade and a half ago, Kragen Javier Sitaker wrote a blog post isinstance() considered harmful, a lot of which I believe holds up pretty well today. It’s well worth reading it in its entirety but the gist is that rather than testing a value against a specific type, you generally wish to check that a value confirms to a particular interface. Or if you prefer, using isinstance is imparting nominative typing in a language where duck typing is the convention. ...

Introduction Typically routes in a web application that require login are explicitly marked as such. Whilst routes that are open to general (anonymous) users, are left unmarked and hence implicitly do not require login. Because the default is to allow all users to visit a particular route, it is easy to forget to mark a route as requiring a login. I’m going to show a small pattern for making sure that all routes in a web application are explicitly marked as either requiring login or not-requiring login. As an example this will be done in a Python, Flask-based web application using Flask-Login but the general idea probably works in at least some other Python web application frameworks. ...

I have a small niggle that comes up in my Python programming. I’m going to describe it and propose a possible addition to the Python programming language that would mostly solve the problem. However, I have not thought through this language modification at all thoroughly so at the moment it’s just a germ of an idea. It wouldn’t be a major change or fix any important problem in any case, it would just solve my own personal peeve. ...

In many cases whilst programming there is a decision to be made as to whether to store some state, or (re)caluate it as and when needed. Obviously every situation is different and therefore there is no one answer which fits. In this post I’m going to attempt to explain the distinction and the benefits/drawbacks of either approach. I hope that just remembering that this choice exists will force me to make an explicit choice, such that I may think about it a bit more. ...

Dougal Matthews has written a blog post detailing how Vulture can be used to find some dead code. For me this was an important reminder not to rely on coverage analysis to detect dead code and remove it from the your maintenance burden. More generally, whilst I adore automated analysis tools that assist the developer in maintaining their code, such automated analysis can give a false sense of completeness, or lead to the developer believing that their code is “good enough”. It is not a problem I have any solution for though. The rest of the post will try to illuminate this view point through the example of dead-code removal. ...

Introduction I do not aim to answer the question in the title more raise the question. The question kind of implies that dynamically typed languages are more productive in at least some ways. It does not imply that statically typed languages are less productive in general, or the opposite. Before going any further, I’m talking about the distinction between static and dynamic typing which is not the same as strong vs weak typing. Static means the type checking is done at compilation before the program is run, whilst dynamic means types are checked whilst the program is running. Not the same as weak vs strong typing and also not the same as explicit vs implicit typing. ...

Selenium is a great way to test web applications and it has Python bindings. I explained in a previous post how to set this up with coverage analysis. However, writing tests is non-trivial, in particular it is easy enough to write tests that suffer from race conditions. Suppose you write a test that includes a check for the existence of a particular DOM element. Here is a convenient method to make doing so a one-liner. It assumes that you are within a class that has the web driver as a member and that you’re using ‘pytest’ but you can easily adapt this for your own needs. ...

Method Cascading Vasudev Ram has a thoughful post about method chaining/cascading that I picked up from planet python in which he basically argues for the use of method cascading. I’m going to disagree. Essentially, I simply don’t understand any benefit of using cascading. It’s a nice post though and includes some references to other method cascading links. Method chaining is the writing of multiple method calls directly after one another, usually on the same line, such as (to take Vasudev’s example): ...

Test First and Mutation Testing I’m going to argue that mutation testing has a strong use in a test first development environment and I’ll conclude by proposing a mechanism to link mutation testing to the source code control mechanism to further aid test first development. Test First Just to be clear, when I say ’test first’ I mean development in which before writing a feature, or fixing a bug, you first write a test which should only pass once you have completed that feature. For the purposes of this post you needn’t be doing that for every line of code you write. The idea here applies whether you are writing the odd feature by first writing a test for it, or whether you have a strict policy of writing no code until there is a test for it. ...

Placement of Python Import Statements Pep 8 specifies that all import statements should be “put at the top of the file, just after any module comments and docstrings, and before module globals and constants.” However, it does not really specify the logic behind this. I’m going to try to articulate some reasons to have import statements somewhere other than directly at the top of the file. I’ll also state some arguments for having such import statements at the top. ...

Update: Flask+Coverage Analysis In a previous post I demonstrated how to get coverage analysis working for a Flask web application in a relatively simple manner. In the section “At then end of your tests” I stated that you needed your tests to clean-up by telling the server to shutdown. The end of your test code would look something like this: finally: driver.get(get_url('shutdown')) ... This could have made things a little fiddly since your test code would have to make sure to access the shutdown route exactly once, regardless of how many tests were run. ...

Flask + Coverage Analysis This post demonstrates a simple web application written in Flask with coverage analysis for the tests. The main idea should be pretty translatable into most Python web application frameworks. Update I’ve updated this scheme and described the update here. tl;dr If you’re having difficulty getting coverage analysis to work with Flask then have a look at my example repository. The main take away is that you simply start the server in a process of its own using coverage to start it. However, in order for this to work you have to make sure you can shut the server process down from the test process. To do this we simply add a new “shutdown” route which is only available under testing. Your test code, whether written in Python or, say Javascript, can then make a request to this “shutdown” route once it completes its tests. This allows the server process to shutdown naturally and therefore allow ‘coverage’ to complete. ...

Suppose you have a web service using a Python web framework such as Flask. So you wish to do a full user test by automating the browser. Should you use the Python bindings to Selenium or CasperJS? In this post I wish to detail some of the advantages and drawbacks of both. Python + Selenium Setup This is fairly straightforward. You are simply writing some Python code that happens to call a library which binds to Selenium. In order for this to work you will need to install phantomJS but using npm this is pretty trivial. ...