Bug reporting is an art form that is too often neglected in software projects. Bug reports allow contributors to participate without deep technical knowledge and at the same time provide a crucial space for developers to be made aware of issues with their software that they could not have foreseen or found themselves, for lack of resources, variety or imagination.Prior art
Unfortunately, there are rarely good guidelines for submitting bug reports. Historically, people have pointed towards How to report bugs effectively or How to ask questions the smart way. While those guides can be useful for motivated people and may seem attractive references for project managers, they suffer from serious issues:
- they are written by technical people, for non-technical people
- as a result, they have a deeply condescending attitude such as calling people "stupid" or various animal names like "mongoose"
- they are also very technical themselves: one starts with a copyright notice and a changelog, the other uses magic words like "Core dumps" and $Id$
- they are too long: sgtatham's is about 3600 words long, esr's is even longer at about 11800 words. those texts will take about 20 to 60 minutes to read by an average reader, according to research
Individual projects have their own guides as well. Linux has the REPORTING_BUGS file that is a much shorter 1200 that can be read under 5 minutes, provided that you can understand the topic at hand. Interestingly, that guide refers to both esr's and sgtatham's guidelines, which means, in the degenerate case where the user hasn't had the "privilege" of reading esr's prose already, they will have an extra hour and a half of reading to do to have honestly followed the guidelines before reporting the bug.
I often find good documentation in the Tails project. Their bug reporting guidelines are easily accessible and quick to read, although they still might be too technical. It could be argued that you need to get technical at some point to get that information out, of course.
In the Monkeysign project, I have started a bug reporting guide that doesn't yet address all those issues. I am considering writing a new guide, but I figured I would look at other people's work and get feedback before writing my own standard.What's the point?
Why have those documents been written? Are people really expected to read them before seeking help? It seems to me unlikely that someone would:
- be motivated enough to do something about a broken part of their computer
- figure out they can do something about it
- read a fifteen thousand words novel about how to report a bug...
- just to finally write a 20-line bug report that has no warranty of support attached to it
And if I would be a paying customer, I wouldn't want to be forced to waste my time reading that prose either: it's your job to help me fix your broken things, not the reverse. As someone doing consulting these days: I totally understand: it's not you, the user, it's us, the developers, that have a problem. We have been socialized through computers, and it makes us weird and obtuse, but that's no excuse, and we need to clean up our act.
Furthermore, it's surprising how often we get (and make!) bug reports that can be difficult to use. The Monkeysign project is very "technical" and I have expected that the bug reports I would get would be well written, with ways to reproduce and so on, but it happened that I received bug reports that were all over the place, didn't have any ways of reproducing or were simply incomplete. Those three bug reports were filed by people that I know to be very technically capable: one is a fellow Debian developer, the second had filed a good bug report 5 days before, and the third one is a contributor that sent good patches before.
In all three cases, they knew what they were doing. Those three people probably read the guidelines mentioned in the past. They may have even read the Monkeysign bug reporting guidelines as well. I can only explain those bug reports by the lack of time: people thought the issue was obvious, that it would get fixed rapidly because, obviously, something is broken.
We need a better way.The takeaway
What are those guides trying to tell us?
- ask questions in the right place
- search for similar questions and issues before reporting the bug
- try to make the developers reproduce the issues
- failing that, try to describe the issue as well as you can
- write clearly, be specific and verbose yet concise
There are obviously contradictions in there, like sgtatham telling us to be verbose and esr tells us to, basically, not be verbose. There is definitely a tension in there, and there are many, many more details about how great bug reports can be if done properly.
I tend towards the side of terseness in our descriptions: people that will know how to be concise will be, people that don't will most likely not learn by reading a 12 000 words novel that, in itself, didn't manage to be parsimonious.
But I am willing to allow for verbosity in bug reports: I prefer too many details instead of missing a key bit of information.Issue trackers
Step 1 is our job: we should send people in the right place, and give them the right tools. Monkeysign used to manage bugs with bugs-everywhere and this turned out to be a terrible idea: you had to understand git and bugs-everywhere to file any bug reports. As a result, there were exactly zero bug reports filed by non-developers during the whole time BE was used, although some bugs were filed in the Debian Bugtracker.
So have a good bug tracker. A mailing list or email address is not a good bug tracker: you lose track of old issues, and it's hard for newcomers to search the archives. It does have the advantage of having a unified interface for the support forum and bug tracking, however.
Redmine, Gitlab, Github and others are all decent-enough bug trackers. The key point is that the issue tracker should be publicly available, and users should be able to register easily to file new issues. You should also be able to mass-edit tickets and users should be able to discover the tracker's features easily. I am sorry to say that the Debian BTS somewhat falls short on those two features.
Step 2 is a shared responsibility: there should be an easy way to search for issues, and we should help the user looking for similar issues. Stackexchange sites do an excellent job at this, by automatically searching for similar questions while you write your question, suggesting similar ones in an attempt to weed out duplicates. Duplicates still happen, but they can then clearly be marked and linked with a distinct mechanism. Most bug trackers do not offer such high level functionality, but should, so I feel the fault lies more on "our" end than at the user's end.Reproducing the environment
Step 3 and 4 are more or less the user's responsibility. We can detail in our documentation how to clearly share the environment where we reproduced the bug, for example, but in the end, the user decides if they want to share that information or not.
In Monkeysign, I have finally implemented joeyh's suggestion of shipping the test suite with the program. I can now tell people to run the test suite in their environment to see if this is a regression that is specific to their environment - so a known bug, in a way - or a novel bug for which I can look at writing a new unit test. I also include way more information about the environment in the --version output, an idea I brought forward in the borg project to ease debugging. That way, people can just send the output of monkeysign --test and monkeysign --version, and I have a very good overview of what is happening on their end. Of course, Monkeysign also supports the usual --verbose and --debug flag that users should enable when reproducing issues.
Another idea is to report bugs directly from the application. We have all seen Firefox or other software have automatic bug reporting tools, but somehow those seem unsatisfactory for a user: we have no feedback of where the report goes, if it's followed up on. It is useful for larger project to get statistical data, but not so useful for users in the short term.
Monkeysign tries to handle exceptions in the code in a graceful way, but could do better. We use a small library to handle exceptions, but that library has since then been improved to directly file bugs against the Github project. This assumes the user is logged into Github, but it is nice to pre-populate bug reports with the relevant information up front.Issue templates
In the meantime, to make sure people provide enough information, I have now moved a lot of the bug reporting guidelines to a separate issue template. That issue template is available through the issue creation form now, although it is not enabled by default, a weird limitation of Gitlab. Issue templates are available in Gitlab and Github.
Issue templates somewhat force users in a straight jacket: there is already something to structure their bug report. Those could be distinct form elements that had to be filled in, but I like the flexibility of the template, and the possibility for users to just escape the formalism and just plead for help in their own way.Issue guidelines
In the end, I opted for a short few paragraphs in the style of the Tails documentation, including a reference to sgtatham, as an optional future reference:
The first aim of a bug report is to tell the developers exactly how to reproduce the failure, so try to reproduce the issue yourself and describe how you did that.
If that is not possible, try to describe what went wrong in detail. Write down the error messages, especially if they have numbers.
Take the necessary time to write clearly and precisely. Say what you mean, and make sure it cannot be misinterpreted.
Include the output of monkeysign --test, monkeysign --version and monkeysign --debug in your bug reports. See the issue template for more details about what to include in bug reports.
If you wish to read more about issues regarding communication in bug reports, you can read How to Report Bugs Effectively, which takes around 20 to 30 minutes.
In the end, there is a fundamental issue with reporting bugs: it assumes our users are literate and capable of writing amazing prose that we will enjoy reading as the last J.K. Rowling novel (if you're into that kind of thing). It's just an unreasonable expectation: some of your users don't even speak the same language as you, let alone read or write it. This makes for challenging collaboration, to say the least. This is where automated reporting makes sense: it doesn't require user's intervention, and the communication is mediated by machines without human intervention and their pesky culture.
But we should, as maintainers, "be liberal in what we accept and conservative in what we send". Be tolerant, and help your users in fixing their issues. It's what you are there for, after all.
And in the end, we all fail the same way. In an attempt to improve the situation on bug reporting guides, I seem to have myself written a 2000 short story that will have taken up a hopefully pleasant 10 minutes of your time at minimum. Hopefully I will have succeeded at being clear, specific, verbose and concise all at once and look forward to your feedback on how to improve our bug reporting culture.
Just in time for the returns of a new season of meetup we've organized and found speakers that we think will open you Python mind. As previously announced, this time we are visiting our friends from Ubisoft Montreal, they will start the evening by showcasing their usage of our favorite language and then introduced one of their key technology based on Python.
We'll also invite 2 other presenters this time: Rami will tell us how we could build amazing microservices with Python, Docker and Kubernetes. Roberto will then show us to scrape the web with the well known and very powerful Selenium library.
Let us know if you are planning to attend on our meetup page: http://www.meetup.com/fr-FR/Montreal-Python/events/233763249/, it always help in our planning.
Thanks to our sponsors we are planning to serve some food.Presentations
Benoit Gagnon: Flare, Ubisoft’s Scalable Video Transcoding Platform
We’ll zoom in on a few Python modules of Flare, Ubisoft’s internal and globally distributed video reviewing and collaboration platform. We’ll see how Python, FFMPEG, Numpy, Redis and Twisted can work together to create a robust and scalable backend using portable Python code. Caution: a gratuitous use of cat videos may or may not be featured in the demos.
Rami Sayar: Building Python Microservices with Docker and Kubernetes
Python is powering your production apps and you are struggling with the complexity, bugs and feature requests you need. You just don't know how to maintain your app anymore. You're scared you have created the kraken that will engulf your entire development team!
Microservices architecture has existed for as long as monolithic applications became a common problem. With the DevOps revolution, it is the time to seriously consider building microservice architectures with Python. This talk will share strategies on how to split up your monolithic apps and show you how to deploy Python microservices using Docker. We will get hands-on with a sample app, walk step-by-step on how to change the app's architecture and deploy it to the cloud.
No longer shall you deal with the endless complexities of monolithic Python apps. Fear the kraken no more!
Roberto Rocha: Selenium for scraping
Selenium is a package used to test websites with its versatile web driver. But it's also very useful as a data scraping tool when dealing with dynamic websites and user input. It allows you to simulate a user by submitting forms and clicking buttons. It's also handy for fetching web elements in multiple ways, whereto by CSS selector or XPath. I propose to demonstrate how to scrape the Soquij legal document website using Selenium.Where
Montréal, H2T 1V4
Tuesday, September 27th 2016 at 6pmSchedule
- 6:00pm — Doors open
- 6:30pm — Presentations start
- 7:30pm — Break
- 7:45pm — Second round of presentations
- 9:00pm — End of the meeting, have a drink with us
- Savoir-faire Linux
September is back and it's for the Montreal Python community to gather again and share exciting new technologies and projects. This month, our friends from Ubisoft are welcoming us into their offices and are going to present to us how they are using Python and how they scaling it at large to powered some of their games.
We are also looking for speakers. It's your opporunity to submit your talk, write us at firstname.lastname@example.org
Ubisoft Offices 5480 St-Dominique Montréal, H2T 1V4
Tuesday, September 27th 2016 at 6pm
The doors are going to be open but you can always let us know if you are coming on our meetup at the following address: http://www.meetup.com/fr-FR/Montreal-Python/events/233763249/
We’d like to thank our sponsors for their continued support:
- Savoir-faire Linux
Join us on August 25th and contribute to the final sprint of Sous-chef, the new management platform of Santropol Roulant. They need our help to get the best tools to manage and deliver food to people with disabilities and difficulties.
This platform will be at the heart of their service and it's entirely based on Django. The project needs volunteers who would like give a hand by contributing to their open platform and pursuing it's development.
It's an opportunity to complete some tasks of the project (https://github.com/savoirfairelinux/santropol-feast), and to get to know what the people of Santropol are doing. Although knowledge of Django is necessary to participate, you don't have to be an expert! Don't be afraid and join us.
Food will be prepared and served by the people from Santropol to the volunteers of the event!
This free event is co-organized by Santropol roulant, Montreal Python, Savoir-Faire Linux, PyLadies Montréal, Montréal Django, .
The event will be bilingual.
Thursday, Auguest 25th, doors are opening at 6pm (Everything start 6:30pm)
Please grab a ticket on our meetup at the following url: https://www.meetup.com/fr-FR/Montreal-Python/events/233348153
Montreal Python developers, this event is for YOU! No matter your favourite programming language, come and grab a burger and a beer in excellent company!
This year, for our traditional inter-community barbecue, we think big! Thanks to the financial and logistic support of our sponsors, we offer you a networking activity you won't soon forget.What's in store
- Huge 225 m² tent with a capacity of 150+ seated guests
- Deck terrace overlooking the Île Notre Dame Lake
- Access to the beach, as well as a private volley-ball court
- 100% beef and vegetarian hamburgers
- Microbrewery beer, GURU and soft drinks
- View of the closing fireworks of L'International des Feux Loto-Québec
- Free parking included
Come one, come all, to an event you won't want to miss!
For more information and tickets, visit the Meetup page.