Skip to main content

External Blogs

Django Carrots Workshop – PyCon Edition!

Montreal Python - Thu, 02/26/2015 - 00:00

Django Carrots – PyCon Edition is a one-­day, intensive, free programming workshop for anyone who wants to learn how to code.

Organized by Geek Girls Carrots (GGC), the Django Carrots curriculum emphasizes individual contact between the student and mentor (before, during and after the workshop), a horizontal structure (students learn from mentors but also from each other), and sharing general knowledge about communities, organization and internet resources to help our participants after the workshop.

How can I sign up?

Please fill out this registration form. Registration is open from February 2nd to March 1st. Everyone who registered should get a response before March 5th. You do not need a ticket for PyCon to attend the Django Carrots workshop during PyCon, but we strongly encourage you to attend the conference as well.

What do I need during the workshop?
  • laptop & charger

Remember: You don’t need to have any additional knowledge about programming to attend (we mean that!). The most important thing is your motivation and readiness to learn!

Who are the mentors?

Our mentors are active programmers working in leading tech companies and universities in Europe and North America. In addition to knowledge and job experience, they are skilled educators who care about diversity and cultural changes in the tech industry.

We look forward to coding with you in Montreal!

Geek Girls Carrots ­is a global community and social enterprise focused on connecting, teaching and inspiring women in Tech and IT. We create community by organizing meetings, workshops, hackathons and other events, gathering people to share their knowledge and experience. We gather in 22 cities, including Warsaw, NYC, London, Berlin, Luxembourg and Sydney.

Categories: External Blogs

PyCon Startup Row - Registration

Montreal Python - Tue, 02/24/2015 - 00:00

Tuesday March 3rd, we're inviting Montreal startups to present their startups to a panel of investors and VCs. Presentations will last 5mn, including a demonstration of the product.

There will be various startups at various stages of growth, from new startups looking for traction to growing startups.

This is a paid event to sustain our costs and to provide appetizers and wine for the networking parts.

Please get your discounted early bird 8$ tickets at https://www.eventbrite.ca/e/mtl-newtech-pycon-edition-tickets-15867698714

for more informations about PyCon: https://us.pycon.org/2015/

First 3 startups to pitch. Others to be announced soon.

  • Elysia: matches you to the trip you need for every occasion. http://www.elysia.co/
  • Warden: Your online business is important. Make sure it stays secure!. https://wardenscanner.com/en/
  • Erudite Science: The toolbox for educational game and app developers. http://eruditescience.com/
  • Zilyo: Compare from over 2,500,000 worldwide listings at once to find the best vacation rental at the best rate. https://zilyo.com/
Agenda:
  • 5:45pm Doors open
  • 6:15pm Event Presentation. Each startup has 5 minutes to pitch, including a demo. Expect 1 or 2 questions from judges
  • 7:30pm End of presentations. Judges deliberate
  • 7:45pm Announcement of the Startup selected for Pycon
  • 7:45pm Stay for networking!

Categories: External Blogs

PyCon Startup Row + MTLNewTech: Call for Applications

Montreal Python - Thu, 02/19/2015 - 00:00

PyCon is partening again with MTL NewTech and us this year to bring one lucky Montreal startup to Pycon, to present alongside with Google, Facebook, Stripe, heroku, Microsoft, Mozilla and many other technology companies.

If you are a startup that meets the requirements below, apply now by filling this form:

http://go.ili.as/PyconContest

With the following information: a) size of the team, b) age of the startup c) your use of Python. Deadline for applications: February 26nd 23h59. Announcement of the startups selected: Starting on February 24th. MTL NewTech Demo & announcement of the winner: March 3rd Feel free to invite fellow startups


==============

Tuesday March 3rd, we're inviting Montreal startups to present their startups to a panel of investors and VCs. Presentations will last 5mn, including a demonstration of the product. There will be various startups at various stages of growth, from new startups looking for traction to growing startups. Startups presenting should be less than 2 years old, less than 15 employees, and must use Python in one way or another (front-end, back-end, etc.).

// Agenda: 5:45pm Doors open 6:15pm Event Presentation. Each startup has 5 minutes to pitch, including a demo. Expect 1 or 2 questions from judges 7:30pm End of presentations. Judges deliberate 7:45pm Announcement of the Startup selected for Pycon 7:45pm Stay for networking!


==============
P.S. Eligible companies must meet the following criteria: Fewer than 15 employees, including founders Less than two years old Use Python somewhere in your startup: backend, frontend, testing, wherever If selected, please confirm that you will staff your booth in the Expo Hall on your appointed day. We will try accommodate your preferences: Friday or Saturday * No repeats. If you were on startup row in a previous year, please give another a startup a chance this year.
==============

Join us on march 3rd if you want to assist to the event: http://www.meetup.com/mtlnewtech/events/220568217/

Categories: External Blogs

PyCon 2015: The Young Coder: Let's Learn Python

Montreal Python - Tue, 02/10/2015 - 00:00

We're once again excited to offer our free tutorials for kids! Called "The Young Coder: Let's Learn Python", we invite kids 12 and over to join us for a day of learning how to program using Python.

We first offered the Young Coders tutorial at PyCon 2013 in Santa Clara and it was an immediate hit. The followup blog post from that event is the most popular post in our blog's history. Kids came from all over to attend it. One flew from South Africa and helped teach other kids. I gave my laptop to a girl who took the tutorial and wanted to attend Richard Jones' PyGame tutorial the next day. Overall, it was so awesome to see kids so excited about learning, about computers, and about Python.

Last year, our second year offering the tutorial and our first year in Montréal, the tutorial was offered both in French and English. We're going to be offering the same this year: a French version of the tutorial will take place on Saturday April 11, and an English version takes place the next day, on Sunday April 12.

Whether you're bringing your kids along for the trip to Montréal or you're local and want to expose your kids to programming, this is a great way to get them interested. The curriculum has now been taught and refined at several events over the last several years, and the kids love it. After learning the basics of Python, through the basic data types and loops and comparisons, the end result is creating a game using the PyGame library. Having been around the room when the tutorial ends, it's one of the best times of PyCon, to see a room full of bright eyed kids who just learned how to tell a computer what to do.

Would your kids be interested in this? Are you a kid who is interested in this? Sign up today! (If you're actually a kid, have a parent sign you up)

http://www.eventbrite.com/e/pycon-2015-young-coders-tickets-15264181578

The tutorials start at 9 AM, break for lunch around 1 PM, and wrap up around 4 PM. Registration is limited to please sign up if you're interested.

See the full details of the program at https://us.pycon.org/2015/events/letslearnpython/

Categories: External Blogs

Migrating from Drupal to Ikiwiki

Anarcat - Fri, 02/06/2015 - 18:02

TLPL; j'ai changé de logiciel pour la gestion de mon blog.

TLDR; I have changed my blog from Drupal to Ikiwiki.

Note: since this post uses ikiwiki syntax (i just copied it over here), you may want to read the original version instead of this one.

will continue operating for a while to
give a chance to feed aggregators to catch that article. It will also
give time to the Internet archive to catchup with the static
stylesheets (it turns out it doesn't like Drupal's CSS compression at
all!) An archive will therefore continue being available on the
internet archive for people that miss the old stylesheet.

Eventually, I will simply redirect the anarcat.koumbit.org URL to
the new blog location, . This will likely be my
last blog post written on Drupal, and all new content will be
available on the new URL. RSS feed URLs should not change.

Why

I am migrating away from Drupal because it is basically impossible to
upgrade my blog from Drupal 6 to Drupal 7. Or if it is, I'll have to
redo the whole freaking thing again when Drupal 8 comes along.

And frankly, I don't really need Drupal to run a blog. A blog was
originally a really simple thing: a web blog. A set of articles
written on the corner of a table. Now with Drupal, I can add
ecommerce, a photo gallery and whatnot to my blog, but why would I do
that? and why does it need to be a dynamic CMS at all, if I get so
little comments?

So I'm switching to ikiwiki, for the following reason:

  • no upgrades necessary: well, not exactly true, i still need to
    upgrade ikiwiki, but that's covered by the Debian package
    maintenance and I only have one patch to it, and there's no data migration! (the last such migration in ikiwiki was in 2009 and was fully supported)
  • offline editing: this is a a big thing for me: i can just note
    things down and push them when I get back online
  • one place for everything: this blog is where I keep my notes, it's
    getting annoying to have to keep track of two places for that stuff
  • future-proof: extracting content from ikiwiki is amazingly
    simple. every page is a single markdown-formatted file. that's it.

Migrating will mean abandoning the
barlow theme, which was
seeing a declining usage anyways.

What

So what should be exported exactly. There's a bunch of crap in the old
blog that i don't want: users, caches, logs, "modules", and the list
goes on. Maybe it's better to create a list of what I need to extract:

  • nodes
    • title ([[ikiwiki/directive/meta]] title and guid tags, guid to avoid flooding aggregators)
    • body (need to check for "break comments")
    • nid (for future reference?)
    • tags (should be added as \[[!tag foo bar baz]] at the bottom)
    • URL (to keep old addresses)
    • published date ([[ikiwiki/directive/meta]] date directive)
    • modification date ([[ikiwiki/directive/meta]] updated directive)
    • revisions?
    • attached files
  • menus
    • RSS feed
    • contact
    • search
  • comments
    • author name
    • date
    • title
    • content
  • attached files
    • thumbnails
    • links
  • tags
    • each tag should have its own RSS feed and latest posts displayed
When

Some time before summer 2015.

Who

Well me, who else. You probably really don't care about that, so let'S
get to the meat of it.

How

How to perform this migration... There are multiple paths:

  • MySQL commandline: extracting data using the commandline mysql tool (drush sqlq ...)
  • Views export: extracting "standard format" dumps from Drupal and
    parse it (JSON, XML, CSV?)

Both approaches had issues, and I found a third way: talk directly to
mysql and generate the files directly, in a Python script. But first,
here are the two previous approaches I know of.

MySQL commandline

LeLutin switched using MySQL requests,
although he doesn't specify how content itself was migrated. Comments
importing is done with that script:

echo "select n.title, concat('| [[!comment format=mdwn|| username=\"', c.name, '\"|| ip=\"', c.hostname, '\"|| subject=\"', c.subject, '\"|| date=\"', FROM_UNIXTIME(c.created), '\"|| content=\"\"\"||', b.comment_body_value, '||\"\"\"]]') from node n, comment c, field_data_comment_body b where n.nid=c.nid and c.cid=b.entity_id;" | drush sqlc | tail -n +2 | while read line; do if [ -z "$i" ]; then i=0; fi; title=$(echo "$line" | sed -e 's/[ ]\+|.*//' -e 's/ /_/g' -e 's/[:(),?/+]//g'); body=$(echo "$line" | sed 's/[^|]*| //'); mkdir -p ~/comments/$title; echo -e "$body" > ~/comments/$title/comment_$i._comment; i=$((i+1)); done

Kind of ugly, but beats what i had before (which was "nothing").

I do think it is the good direction to take, to simply talk to the
MySQL database, maybe with a native Python script. I know the Drupal
database schema pretty well (still! this is D6 after all) and it's
simple enough that this should just work.

Views export

[[!img 2015-02-03-233846_1440x900_scrot.png class="align-right" size="300x" align="center" alt="screenshot of views 2.x"]]

mvc recommended views data export on Lelutin's
blog. Unfortunately, my experience with the views export interface has
been somewhat mediocre so far. Yet another reason why I don't like
using Drupal anymore is this kind of obtuse dialogs:

I clicked through those for about an hour to get JSON output that
turned out to be provided by views bonus instead of
views_data_export. And confusingly enough, the path and
format_name fields are null in the JSON output
(whyyy!?). views_data_export unfortunately only supports XML,
which seems hardly better than SQL for structured data, especially
considering I am going to write a script for the conversion anyways.

Basically, it doesn't seem like any amount of views mangling will
provide me with what i need.

Nevertheless, here's the [[failed-export-view.txt]] that I was able to
come up with, may it be useful for future freedom fighters.

Python script

I ended up making a fairly simple Python script to talk directly to
the MySQL database.

The script exports only nodes and comments, and nothing else. It makes
a bunch of assumptions about the structure of the site, and is
probably only going to work if your site is a simple blog like mine,
but could probably be improved significantly to encompass larger and
more complex datasets. History is not preserved so no interaction is
performed with git.

Generating dump

First, I imported the MySQL dump file on my local mysql server for easier
development. It is 13.9MiO!!

mysql -e 'CREATE DATABASE anarcatblogbak;' ssh aegir.koumbit.net "cd anarcat.koumbit.org ; drush sql-dump" | pv | mysql anarcatblogbak

I decided to not import revisions. The majority (70%) of the content has
1 or 2 revisions, and those with two revisions are likely just when
the node was actually published, with minor changes. ~80% have 3
revisions or less, 90% have 5 or less, 95% 8 or less, and 98% 10 or
less. Only 5 articles have more than 10 revisions, with two having the
maximum of 15 revisions.

Those stats were generated with:

SELECT title,count(vid) FROM anarcatblogbak.node_revisions group by nid;

Then throwing the output in a CSV spreadsheet (thanks to
mysql-workbench for the easy export), adding a column numbering the
rows (B1=1,B2=B1+1), another for generating percentages
(C1=B1/count(B$2:B$218)) and generating a simple graph with
that. There were probably ways of doing that more cleanly with R,
and I broke my promise to never use a spreadsheet again, but then
again it was Gnumeric and it's just to get a rough idea.

There are 196 articles to import, with 251 comments, which means an
average of 1.15 comment per article (not much!). Unpublished articles
(5!) are completely ignored.

Summaries are also not imported as such (break comments are
ignored) because ikiwiki doesn't support post summaries.

Calling the conversion script

The script is in [[drupal2ikiwiki.py]]. It is called with:

./drupal2ikiwiki.py -u anarcatblogbak -d anarcatblogbak blog -vv

The -n and -l1 have been used for first tests as well. Use this
command to generate HTML from the result without having to commit and
push all:

ikiwiki --plugin meta --plugin tag --plugin comments --plugin inline . ../anarc.at.html

More plugins are of course enabled in the blog, see the setup file for
more information, or just enable plugin as you want to unbreak
things. Use the --rebuild flag on subsequent runs. The actual
invocation I use is more something like:

ikiwiki --rebuild --no-usedirs --plugin inline --plugin calendar --plugin postsparkline --plugin meta --plugin tag --plugin comments --plugin sidebar . ../anarc.at.html

I had problems with dates, but it turns out that I wasn't setting
dates in redirects... Instead of doing that, I started adding a
"redirection" tag that gets ignored by the main page.

Files and old URLs

The script should keep the same URLs, as long as pathauto is enabled
on the site. Otherwise, some logic should be easy to add to point to
node/N.

To redirect to the new blog, rewrite rules, on original blog, should
be as simple as:

Redirect / http://anarc.at/blog/

When we're sure:

Redirect permanent / http://anarc.at/blog/

Now, on the new blog, some magic needs to happen for files. Both
/files and /sites/anarcat.koumbit.org/files need to resolve
properly. We can't use symlinks because
ikiwiki drops symlinks on generation.

So I'll just drop the files in /blog/files directly, the actual
migration is:

cp $DRUPAL/sites/anarcat.koumbit.org/files $IKIWIKI/blog/files rm -r .htaccess css/ js/ tmp/ languages/ rm foo/bar # wtf was that. rmdir * sed -i 's#/sites/anarcat.koumbit.org/files/#/blog/files/#g' blog/*.mdwn sed -i 's#http://anarcat.koumbit.org/blog/files/#/blog/files/#g' blog/*.mdwn chmod -R -x blog/files sudo chmod -R +X blog/files

A few pages to test images:

  • http://anarcat.koumbit.org/node/157
  • http://anarcat.koumbit.org/node/203

There are some pretty big files in there, 10-30MB MP3s - but those are
already in this wiki! so do not import them!

Running fdupes on the result helps find oddities.

The meta guid directive is used to keep the aggregators from finding
duplicate feed entries. I tested it with Liferea, but it may freak out
some other sites.

Remaining issues
  • postsparkline and calendar archive disrespect meta(date)
  • merge the files in /communication with the ones in /blog/files
    before import
  • import non-published nodes
  • check nodes with a format different than markdown (only a few 3=Full
    HTML found so far)
  • replace links to this wiki in blog posts with internal links

More progress information in [[the script|drupal2ikiwiki.py]] itself.

Categories: External Blogs

Montréal-Python 51: Ostentatory Pagination

Montreal Python - Sun, 02/01/2015 - 00:00

After a long holiday break, Montréal-Python returns with a special themed meeting on data. Data is the core of our job and makes up for one of the most dynamic areas in the Python world.

So we've put together a meeting to give you a good overview of Montréal's pythonista "Data Science" scene.

If you would like to discuss online about our meetings or with our speakers, please take a look at our brand new meetup.com page: http://www.meetup.com/Montreal-python.

Evening Program:

Marcin Swiatek: Introduction to the statsmodel library

(http://statsmodels.sourceforge.net/)

I have been using scipy.stats as my main library for all things statistics. And it was good and understandable. Recently, however, I needed to dive into some more advanced data exploration. Something people normally would use R for. Enter statsmodels. I think I got just enough hang of it to talk about it for few minutes and give the neophyte's account of its strengths and uses. We will work with an example data set to try and find something interesting in it...

Louis-Philippe Huberdeau: Les défis de l'intégration python 2.x et python 3.x avec pyZMQ

(https://wardenscanner.com/)

Présentation de l'architecture pyZMQ ainsi que des défis lors de l'interopérabilité avec des composantes vieillissantes.

Marc Tardif: R, shiver me Python

(http://www.immun.io/)

A Python project is more than just code; it's a lot of stuff like tests to make sure it works, documentation to describe how it works and graphs to measure how well it works. This presentation will demonstrate using R to visualize metrics produced by a project.

David Taylor: A day in the life of a Data Science Pythonista

(http://www.prooffreader.com/)

David Taylor of dtdata.io and prooffreader.com explains some of his personal data science projects, as well as the professional data science projects he does in order to afford to do personal data science projects.

When:

Monday, the February 9th 2015

Where:

UQÀM, Pavillion PK

201, avenue du Président-Kennedy

Room PK-1140

Comment

It's free, just join us!

Schedule:
  • 6:00pm — Doors open
  • 6:30pm — Presentations start
  • 7:30pm — Break
  • 7:45pm — Second round of presentations
  • 9:00pm — One free beer offered at Bénélux just across the street
We’d like to thank our sponsors for their continuous support:
  • UQÀM
  • Bénélux
  • w.illi.am/
  • Outbox
  • Savoir-Faire Linux
  • Caravan
  • iWeb
Categories: External Blogs

Call for speakers for Montréal-Python 51: Ostentatory Pagination

Montreal Python - Thu, 01/15/2015 - 00:00

It is the start of a new year and we've decided that the data science will be honored at our next meetup, the first monthly event of 2015. If you are using Python to manipulate, change, vizualise, analyse or produce data, we would love to hear from you.

We are looking for speakers for short and longer presentation (5-45mins).

If you are willing to take this opportunity and come show us what you are doing, send us a blurb and give a small introduction to what you are doing at the following email address:

mtlpyteam@googlegroups.com

When:

Monday, the February 9st 2015

Schedule:
  • 6:00pm — Doors open
  • 6:30pm — Presentations start
  • 7:30pm — Break
  • 7:45pm — Second round of presentations
  • 9:00pm — One free beer offered at Bénélux just across the street
We’d like to thank our sponsors for their continuous support:
  • UQÀM
  • Bénélux
  • w.illi.am/
  • Outbox
  • Savoir-Faire Linux
  • Caravan
  • iWeb
Categories: External Blogs

Montreal-Python: Project Nights 9

Montreal Python - Thu, 01/08/2015 - 00:00

It is the beginning of 2015, and it is a great way to start the new year with a night of coding a unique opportunity to share ideas and do some networking.

We are inviting you to our latest Project Night in the hearth of the Mile-End, at the office of our friends from Plotly.

What to expect:

We will begin with a short presentation of the graphs tools developed by the team at Plotly. They will show us some magic they are doing with our favorite language and how to play with their api.

Then, we have a 3 projects that will be represented:

If you have a project, if you need help on a piece of code you are working on, just come and there will be other pythonisthas there to share with.

When:

On Thursday, January 22 at 6pm

Where:

At the office of our friends from Plotly (https://plot.ly/)

5555 avenue de Gaspé, Suite 201, Montreal http://www.openstreetmap.org/?mlat=45.52760&mlon=-73.59880#map=18/45.52760/-73.59880

How:

Please register on our eventbrite event at the following address: http://www.eventbrite.ca/e/montreal-python-project-nights-9-tickets-15212345535

Categories: External Blogs

The future of Python Nights

Montreal Python - Tue, 12/02/2014 - 00:00

After more then a dozen python nights, I'm glad to announce the success of the event.

Categories: External Blogs

bup vs attic silly benchmark

Anarcat - Tue, 11/18/2014 - 00:39

after see attic introduced in a discussion about bup, i figured out i could give it a try. it was answering two of my biggest concerns with bup:

  • backup removal
  • encryption

and seemed to magically out of nowhere and basically do everything i need, with an inline manual on top of it.

disclaimer

Note: this is not a real benchmark! i would probably need to port bup and attic to liw's seivot software to report on this properly (and that would amazing and really interesting, but it's late now). even worse, this was done on a production server with other stuff going on so take results with a grain of salt.

procedure and results

Here's what I did. I setup backups of my ridiculously huge ~/src directory on the external hard drive where I usually make my backups. I ran a clean backup with attic, than redid it, then I ran a similar backup with bup, then redid it. Here are the results:

anarcat@marcos:~$ sudo apt-get install attic # this installed 0.13 on debian jessie amd64 [...] anarcat@marcos:~$ attic init /mnt/attic-test: Initializing repository at "/media/anarcat/calyx/attic-test" Encryption NOT enabled. Use the "--encryption=passphrase|keyfile" to enable encryption. anarcat@marcos:~$ time attic create --stats /mnt/attic-test::src ~/src/ Initializing cache... ------------------------------------------------------------------------------ Archive name: src Archive fingerprint: 7bdcea8a101dc233d7c122e3f69e67e5b03dbb62596d0b70f5b0759d446d9ed0 Start time: Tue Nov 18 00:42:52 2014 End time: Tue Nov 18 00:54:00 2014 Duration: 11 minutes 8.26 seconds Number of files: 283910 Original size Compressed size Deduplicated size This archive: 6.74 GB 4.27 GB 2.99 GB All archives: 6.74 GB 4.27 GB 2.99 GB ------------------------------------------------------------------------------ 311.60user 68.28system 11:08.49elapsed 56%CPU (0avgtext+0avgdata 122824maxresident)k 15279400inputs+6788816outputs (0major+3258848minor)pagefaults 0swaps anarcat@marcos:~$ time attic create --stats /mnt/attic-test::src-2014-11-18 ~/src/ ------------------------------------------------------------------------------ Archive name: src-2014-11-18 Archive fingerprint: be840f1a49b1deb76aea1cb667d812511943cfb7fee67f0dddc57368bd61c4bf Start time: Tue Nov 18 00:05:57 2014 End time: Tue Nov 18 00:06:35 2014 Duration: 38.15 seconds Number of files: 283910 Original size Compressed size Deduplicated size This archive: 6.74 GB 4.27 GB 116.63 kB All archives: 13.47 GB 8.54 GB 3.00 GB ------------------------------------------------------------------------------ 30.60user 4.66system 0:38.38elapsed 91%CPU (0avgtext+0avgdata 104688maxresident)k 18264inputs+258696outputs (0major+36892minor)pagefaults 0swaps anarcat@marcos:~$ sudo apt-get install bup # this installed bup 0.25 anarcat@marcos:~$ free && sync && echo 3 | sudo tee /proc/sys/vm/drop_caches && free # flush caches anarcat@marcos:~$ export BUP_DIR=/mnt/bup-test anarcat@marcos:~$ bup init Dépôt Git vide initialisé dans /mnt/bup-test/ anarcat@marcos:~$ time bup index ~/src Indexing: 345249, done. 56.57user 14.37system 1:45.29elapsed 67%CPU (0avgtext+0avgdata 85236maxresident)k 699920inputs+104624outputs (4major+25970minor)pagefaults 0swaps anarcat@marcos:~$ time bup save -n src ~/src Reading index: 345249, done. bloom: creating from 1 file (200000 objects). bloom: adding 1 file (200000 objects). bloom: creating from 3 files (600000 objects). Saving: 100.00% (6749592/6749592k, 345249/345249 files), done. bloom: adding 1 file (126005 objects). 383.08user 61.37system 10:52.68elapsed 68%CPU (0avgtext+0avgdata 194256maxresident)k 14638104inputs+5944384outputs (50major+299868minor)pagefaults 0swaps anarcat@marcos:attic$ time bup index ~/src Indexing: 345249, done. 56.13user 13.08system 1:38.65elapsed 70%CPU (0avgtext+0avgdata 133848maxresident)k 806144inputs+104824outputs (137major+38463minor)pagefaults 0swaps anarcat@marcos:attic$ time bup save -n src2 ~/src Reading index: 1, done. Saving: 100.00% (0/0k, 1/1 files), done. bloom: adding 1 file (1 object). 0.22user 0.05system 0:00.66elapsed 42%CPU (0avgtext+0avgdata 17088maxresident)k 10088inputs+88outputs (39major+15194minor)pagefaults 0swaps

Disk usage is comparable:

anarcat@marcos:attic$ du -sc /mnt/*attic* 2943532K /mnt/attic-test 2969544K /mnt/bup-test

People are encouraged to try and reproduce those results, which should be fairly trivial.

Observations

Here are interesting things I noted while working with both tools:

  • attic is Python3: i could compile it, with dependencies, by doing apt-get build-dep attic and running setup.py - i could also install it with pip if i needed to (but i didn't)
  • bup is Python 2, and has a scary makefile
  • both have an init command that basically does almost nothing and takes little enough time that i'm ignoring it in the benchmarks
  • attic backups are a single command, bup requires me to know that i first want to index and then save, which is a little confusing
  • bup has nice progress information, especially during save (because when it loaded the index, it knew how much was remaining) - just because of that, bup "feels" faster
  • bup, however, lets me know about its deep internals (like now i know it uses a bloom filter) which is probably barely understandable by most people
  • on the contrary, attic gives me useful information about the size of my backups, including the size of the current increment
  • it is not possible to get that information from bup, even after the fact - you need to du before and after the backup
  • attic modifies the files access times when backing up, while bup is more careful (there's a pull request to fix this in attic, which is how i found out about this)
  • both backup systems seem to produce roughly the same data size from the same input
Summary

attic and bup are about equally fast. bup took 30 seconds less than attic to save the files, but that's not counting the 1m45s it took indexing them, so on the total run time, bup was actually slower. attic is also (almost) two times faster on the second run as well. but this could be within the margin of error of this very quick experiment, so my provisional verdict for now would be that they are about as fast.

bup may be more robust (for example it doesn't modify the atimes), but this has not been extensively tested and is more based with my familiarity with the "conservatism" of the bup team rather than actual tests.

considering all the features promised by attic, it makes for a really serious contender to the already amazing bup.

Next steps

The properly do this, we would need to:

  • include other software (thinking of Zbackup, Burp, ddar, obnam, rdiff-backup and duplicity)
  • bench attic with the noatime patch
  • bench dev attic vs dev bup
  • bench data removal
  • bench encryption
  • test data recovery
  • run multiple backup runs, on different datasets, on a cleaner environment
  • ideally, extend seivot to do all of that

Note that the Burp author already did an impressive comparative benchmark of a bunch of those tools for the burp2 design paper, but it unfortunately doesn't include attic or clear ways to reproduce the results.

Categories: External Blogs

Montréal-Python 50: Monochromatic Necessity

Montreal Python - Mon, 11/17/2014 - 00:00

**UPDATE** Due to a scheluding mix-up our event will be at UQAM

For our 50th edition of our Monthly meetups we wanted to offer you an amazing evening with an amazing lineup. That's why this time we have the chance to be welcome at Notman Housse by our friends from Real Ventures.

We would like to thanks all the sponsors and Datacratic to support us for this special evening.

In the mean time, don't forget to join us on Tuesday for our http://montrealpython.org/en/2014/11/python-night-xiii/

And also don't forget, if you have anything you would like to present or announce, just show up and tell us, we are always happy to give a chance to anyone !

Flash Presentations (5-10 minutes each):

George Peristerakis: How openstack automated the software development process.

(http://enovance.com)

The Openstack system is a collection of open source services that helps you setup a cloud infrastructure. It is also one of the largest and most active code base. I will talk about how openstack automated most of its development process.

David Taylor: Top 10 'Python idioms it took me way too long to figure out

(http://www.prooffreader.com/)

Since I adopted Python after learning programming in Visual Basic, I gradually found out features that, had I known them before, would have saved me from having to reinvent the wheel, like collections.Counter, dict.get(), list.enumerate(), etc.

Slides: http://prooffreaderplus.blogspot.ca/2014/11/top-10-python-idioms-i-wished-id.html

Main Presentations:

Nicolas Kruchten: Data Science and Machine Learning with PyData

(http://datacratic.com/)

Nicolas will walk through a demonstration of ways to explore, visualize and extract insights from data using PyData tools like IPython Notebook, numpy, pandas and scikit-learn.

Slides: http://opensource.datacratic.com/mtlpy50/slides.html

Sources: https://github.com/datacratic/mtlpy50

Video: https://www.youtube.com/watch?v=2lpS6gUwiJQ

Stéphane Guidoin: CKAN plateforme de données ouvertes en python: forces et défis

(http://nordouvert.ca/)

Lancé par l'Open Knowledge Foundation et développé en Python, CKAN a vocation a servir de portail de données ouvertes, notamment pour les gouvernements. CKAN a acquis une position dominante, surtout auprès des gouvernements nationaux. Malgré cette position, le développement de l'outil n'est pas aussi rapide que beaucoup souhaiteraient et plateformes comme Socrata demeurent très présentes, notamment auprès des municipalités, malgré leur prix nettement supérieur. La présentation couvrira l'architecture technologique de CKAN, ce qui lui a permis de devenir une solution dominante, mais aussi les défis de croissance auxquels fait face ce projet.

Cameron Davidson-Pilon: What is PySpark and When Should I Use it?

(http://www.shopify.ca/)

Spark is being called the next generation of Hadoop: it's faster, more accessible and has a large community behind it. PySpark is a Python interface to Spark. In this talk, we'll discuss what MapReduce is, PySparks API, and when to use PySpark versus another tool.

When:

Monday, the December 1st 2014

Where:

UQÀM, Pavillion PK

201, avenue du Président-Kennedy

Room PK-1140

Comment

It's free, just join us !

Schedule:
  • 6:00pm — Doors open
  • 6:30pm — Presentations start
  • 7:30pm — Break
  • 7:45pm — Second round of presentations
  • 9:00pm — One free beer offered at Bénélux just across the street
We’d like to thank our sponsors for their continuous support:
  • Datacratic
  • Real Ventures
  • UQÀM
  • Bénélux
  • w.illi.am/
  • Outbox
  • Savoir-Faire Linux
  • Caravan
  • iWeb
Categories: External Blogs

Project Night XIII

Montreal Python - Tue, 11/11/2014 - 00:00

Welcome to Python Night XIII (hosted by Caravan)

After a long absence, Python Night is back and we are inviting everyone to join us in a night of coding, laughs and all out good time.

What? You don't already know what a Python Night is?

Well, it's a friendly evening where we code, chat about projects (not too loud so people won't code bugs :) ) and help each other. You come, with or without your project and we'll try to something with it.

Already announces projects:

WHEN November 18, 2014 @6pm

WHERE Caravan, 5334 de Gaspé, office #1204 (Montreal)

HOW Just grab your free ticket on: http://python-project-night-13.eventbrite.ca

Bring your computer and your smile, we'll provide beer and pizza !

Categories: External Blogs

Meeting Python and VFX

Montreal Python - Mon, 11/10/2014 - 00:00

On Monday, November the 17th, the NAD Centre will partner up with our friends at [Savoir-Faire Linux to organize a "Python and VFX" meetup. Python is ubiquitous in the world of special effects: from Maya to Softimage and 3D Studio Max, not to mention Blender of course.

The presentations will be held at NAD Centre, 405 rue Ogilvy, 3rd floor, at 6PM.

As you rpobably know, we've already had an event about VFX and Python during MP31:http://montrealpython.org/fr/2012/09/mp31/.

And this one is promissing to be very interesting if you are using any 3D software or if you would like to know more about real case study of useful way of using Python.

Program for the evening:

  • 6:00PM – Welcome
  • 6:30PM – Eric Thivierge, Rigging R&D at Hybride Technologie “Python: Bridging Technologies”
  • 7:00PM – Dave Lajoie, R&D Director at Digital District “Python in a vfx/animation pipeline”
  • 7:45PM – Jordi Riera, Python Software Developer at Savoir-Faire Linux “How to train your Python or how to improve python codes”
  • 8:15PM – Networking
  • 9:00PM – Wrap up

Don't forget to register your place at: http://elite.nad.ca/product/meetup-python-2/

For more informations:

See you there !

Categories: External Blogs

Epic Lameness

Eric Dorland - Mon, 09/01/2008 - 17:26
SF.net now supports OpenID. Hooray! I'd like to make a comment on a thread about the RTL8187se chip I've got in my new MSI Wind. So I go to sign in with OpenID and instead of signing me in it prompts me to create an account with a name, username and password for the account. Huh? I just want to post to their forum, I don't want to create an account (at least not explicitly, if they want to do it behind the scenes fine). Isn't the point of OpenID to not have to create accounts and particularly not have to create new usernames and passwords to access websites? I'm not impressed.
Categories: External Blogs

Sentiment Sharing

Eric Dorland - Mon, 08/11/2008 - 23:28
Biella, I am from there and I do agree. If I was still living there I would try to form a team and make a bid. Simon even made noises about organizing a bid at DebConfs past. I wish he would :)

But a DebConf in New York would be almost as good.
Categories: External Blogs
Syndicate content