mercredi 11 octobre 2017

Why I chose GNU Linux for my professional laptop

Before you ask the question, or even care about it, here is why I chose to put Debian on my new laptop.

I have to say I really do not like Windows. I'm sure the system got better and better among its versions and that Microsoft itself is improving its opensource policy. However, I do not like how Windows looks, I do not like how it works, I feel in jail with it. In my previous job, my screen used to show a terminal with Cygwin and Vim most of the time, helping me to gain a nice nerd reputation!

The classic dilemma then is to choose between buying a Macbook, or install Linux an another laptop.

We have a 2013 iMac at home and I was globally happy with it at first. Everything is simpler with it, the system is stable, the hardware is beautiful. But with the updates, some stuff got annoying:

  • The OS randomly refuses to mount my USB keys
  • Launchpad randomly refuses to allow me to search apps with the keyboard
  • The system takes longer and longer to boot
  • The system won't launch graphical app (even the Finder) from within tmux

Moreover, last releases of Macbooks are sooo expensive, and lots of people complain about hardware quality on Twitter.

I started regretting my previous experience with Debian on my previous PC. On the other hand, I know that with Linux, I can spend a lot of time trying stuff and fix (sometimes non existent) issues.

So I chose to buy a Lenovo ThinkPad, because I saw at Pycon that a LOT of people had one. Chances are that they are Linux friendly (besides, I think I've read somewhere that they're used by people from Red Hat).

I chose Debian because of the free software philosophy behind it, the large choice of packages and the fact that private software are often distributed as Debian packages. I hesitated with Ubuntu. But seeing Amazon ads on the default desktop on Ubuntu decided me.

I completed the installation two weeks ago and everything goes surprisingly well, far beyond my expectations.

For now, I keep the dual boot with Windows, in case I have to use it for work. But I hope to drop the partition soon!

If you're questioning yourself about going on Linux for your professional laptop, here are some advices.

  • Consider buying a Lenvo Thinkpad. They are well supported and you have a customer support. I read complains about Dell XPS, though they are shipped with Ubuntu. I also saw people with Asus Zenbooks, it can be a good alternative.
  • You'll have software and hardware issues for sure. Yet, during my Linux year I always found a solution eventually on the Internet. In the extreme case, you still can boot on the latest release of the kernel. It's not that hard.
  • You can easily taylor a system that fits your needs and helps you to be more efficient, it's priceless when you rely on your laptop to make money.

samedi 7 octobre 2017

Debian on my new ThinkPad T470p

As I started my new activity as a contractor, I bought a new laptop: a Lenovo Thinkpad T470p. Not that I fell in love with Windows, but to install Debian on it. Here is the description of the journey. I leave the reason of my choice in another post.

Hardware

The hardware is made of:

  • A i7 processor
  • 8 GB Ram
  • 256 GB Samsung SSD
  • A Nvidia GeForce 940 mx graphic card
  • Intel 8265 wifi component

I'm quite happy with this.

What you'll need

Prepare this stuff or it won't work:

  • Your laptop
  • An USB key (not a big one, I used a 1 GB old key I had)
  • An Internet connection through ethernet cable, since what I describe won't work by wifi.

Installation

I have several installation of Linux behind me, but only on BIOS based systems and raspberry PI. So I learned about UEFI, not without some difficulties!

I want to install Debian testing, because Debian stable usually contains old stuff. So at the time I write these lines, its little name is Buster and it might be Debian 10th release.

First thing to do is to download an ISO. Do yourself a favour and don't chose a Debian testing installer. It's an alpha version and will simply not work (I used netinst ISO and it doesn't contains the mandatory packages). Go on with stable distribution. I chose netinst, because I did not have a big bootable USB stick at home and I want a minimal installation. I chose an amd64 architecture (though you have an Intel CPU… I know, I know…).

Now you have to make an UEFI bootable USB key. To do so, from Windows 10, I used Rufus. I first made the mistake of choosing to make it BIOS bootable. The thing is that depending the type you choose (UEFI or BIOS), the installer will now if it has to install on UEFI or BIOS. I made the bad choice at first and the laptop did not boot at the end of the installation. So, choose to format the key as a UEFI boot device.

Remember to shrink your disk to have some room for your Linux system. I used the disk management tool provided with Windows 10. I did not gain lot of disk space, but I guess it is the most secure way to do so.

Now reboot having the key plugged and go into the BIOS by pressing Return while booting. In there, disable Secure Boot in security options (I guess). You can also change boot order to make it boot directly from the disk, instead of Windows bootloader. Exit the BIOS and press F12 to access the list of devices to boot from. Choose the USB stick. You now enter the installer.

From here, you won't face a lot of trouble, do whatever you want. Just notice:

  • I prefer the simple ncursed based installer instead of the graphical one. I know I'm a nerd but believe me, it's easier to use.
  • The wifi chip needs a non free firmware that is not provided by the ISO. The installer propose you to get it from another support. Say no.
  • I advise you to keep the installation to its minimum, without a graphical desktop since you will upgrade to testing right after the process.
  • I chose not to use Swap partition. Come on, I have 8GB RAM! It's controversial though, and I could create a swap file later on if things goes wrong.

At the end of the installation, remove the USB key and let the installer reboot the machine for you. You should then see Grub to boot on your Debian.

Post installation

First thing I did was to switch to testing repositories and activate contrib and non-free repo. To do so, replace the content of /etc/apt/sources.list with:

deb http://ftp.fr.debian.org/debian/ testing main contrib non-free
deb-src http://ftp.fr.debian.org/debian/ testing main contrib non-free

deb http://security.debian.org/debian-security testing/updates main contrib non-free
deb-src http://security.debian.org/debian-security testing/updates main contrib non-free

deb http://ftp.fr.debian.org/debian/ testing-updates main contrib non-free
deb-src http://ftp.fr.debian.org/debian/ testing-updates main contrib non-free

Then, as root, launch the upgrade:

apt-get dist-upgrade

Remember to install the wifi firmware with:

apt-get install firmware-iwlwifi

And voilà, you can install whatever you want.

Troubleshooting

I faced a few caveats after the installation, partly because I chose to use XFCE as my desktop environment and lightdm as a display manager (because I like to try different desktop).

The main problem was the inability to log back to my session after locking it. After spending a looong time on Google, I found that it is due to an incompatibility between XFCE4 power management and lightdm-locker, the locking frontend for lightdm. The no brainer here is to install and configure the (ugly) xscreensaver, as XFCE tries to use it before light-locker while locking the session. Its done in a simple shell script (/usr/bin/xflock4), so feel free to check and hack it if you want.

I also found some error messages related to firmware in the syslog, which made me install firmware-misc-nonfree package. I thought at first it was related to the locking problem, but actually I do not know the effect of this fix.

Conclusion

I'm really happy with this system. It's amazingly fast. I took time to install Node and Python 3.6 from sources and I'm ready to hack!

If everything works well in near future, I may drop Windows partition and reinstall the system to use the whole disk and encrypt the partitions. The journey is far from being over!

vendredi 22 septembre 2017

Adopting vim native package management.

OK, today, I switch to Vim 8 native package management.

Before, I used Pathogen with controversial git submodule feature.

I think it's always better to use native features instead of third party ones, even if I never had any issue with Pathogen.

If you have a similar configuration, just type following commands to switch (assuming your modules are in .vim/bundle directory)

cd ~/.vim  # go to your vim config directory
mkdir -p pack/myplugs/start  # create a pack dir for plugin loading on startup
for DIR in bundle/*; do git mv $DIR pack/myplugs/start; done  # move every submodule from bundle to pack keeping git aware 

You may have to update your git submodules if they emptied in the process:

git submodule update --recursive --remote

Then edit your ~/.vimrc file to remoce call to pathogen

" remove this line
call pathogen#infect() 

Rerun vim, to check if everything is OK for you. Cleaning the .vim directory is left as an exercise.

Native package management feature adds plugins to the runtime path at startup for bundles stored in .vim/pack/whatever/start. You can also configure optional module by putting them in .vim/pack/whatever/opt. If you put a module foodebug in that directory, you have to activate it manually with command:

:packadd foodebug

You could achieve the same with Pathogen though.

mercredi 20 septembre 2017

Recaptcha in Django

I've been writing a small website with Django these days. I wanted to put a form with a captcha to prevent an evil hacker from exploding my database quotas. So I looked at the popular Google's Recaptcha service.

According to documentation, when processing the form, you have to send a token to Google and analyze the response to actually validate the form. Well, OK, it's not so complex, but I expected a 2 minutes solution!

Fortunately, Django is easily extensible through third party apps. And an app exists for my reCaptcha purpose.

So the message here is: if you want to add an external service to your website, look at what Django has for you or if a Django app exists for that purpose.

You (are old and you) want to add an RSS / Atom feed? It comes directly with Django!

More on reCaptcha. If your want to see the current reCaptcha widget to let Google guess automagickly if you're a robot, add the option bellow to your settings.py:

NOCAPTCHA = True

And a last word about privacy. Google's recaptcha uses DoubleClick and the fact that Google knows everything on everyone. I chose that because I believe that users are used to it and because of its simplicity. If privacy is an issue for you (it's legitimate), other kind of captcha exists: color based, math based, etc.

dimanche 9 juillet 2017

Feedback loop please

I’ve been working on data transfer processes these days. The transfer time is ridiculously slow regarding the amount of data (about 100k records). By the way it is geographic data, so it might be normal. It irritated me though and made me think about feedback loop.

The importance of shortening the feedback loop between coding and watching the result is huge.

A short feedback loop helps you to achieve more in less time. It also prevents you to get bored and fall into procrastination.

Days spent waiting for feedback are long and make me feel depressed. Days when I get quick feedbacks are pretty cool, whatever I am doing.

Some idea to shorten feedback loop :

  • TDD of course makes you write code that can be tested faster. Actually, taking much time to write test or wait for the test run to finish is a sign that you do something wrong. However TDD is not sufficient to test your whole app and you have to add system tests. They last longer.
  • Avoid network. Your network is unreliable. Use a local database on your machine when it’s possible. It also will prevent to crash the shared database for respect to your colleagues.
  • Automate what can be: to load your dataset, check the initial state of the system, test, cleanup.
  • Reduce the amount of data you will handle during your tests, at least for early stage of development
  • Ask for faster hardware in last resort.

This blog post is can of a truism. At least it will remain a note to myself!

vendredi 14 avril 2017

logging.exception

By skimming Python documentation today, I realized that logging module provides an exception level. Its goal is to show you a stacktrace when you call it from an except block.

You may have noticed that if you log an exception with level error, you only get exception message:

#!/usr/bin/env python3
# -*- coding: utf-8 -*-

import logging

def send_unexpected_exception():
    raise Exception("I did not expect this")

logging.basicConfig(level=logging.DEBUG)

try:
    send_unexpected_exception()
except Exception as e:
    logging.error(e)

Returns:

ERROR:root:I did not expect this

Now use exception level:

It returns:

ERROR:root:I did not expect this
Traceback (most recent call last):
  File "logging_exception.py", line 12, in <module>
    send_unexpected_exception()
  File "logging_exception.py", line 7, in send_unexpected_exception
    raise Exception("I did not expect this")
Exception: I did not expect this

I think it’s an interesting way to provide feedback and context if your app raises an unexpected exception.

mercredi 18 janvier 2017

Accept my ideas

I like to question everything when I land into an existing team. As a normally pretentious person, I like looking smart, making people feel I'm experienced and that they can count on me. Yet, when I say what I find bad in a code base and when I expose my brilliant ideas, my new teammates don't follow me. That hurts my pride so hard!

“we've done this until now, there is no reason to change” says the lead dev. Of course they are reasons change, since the need for software and our knowledge evolve all the time! Questioning things is a good habit.

I've given up trying to look smart. It's being a jerk and lead to nothing. My solutions are meaningless when there is no problem, at least when the effects of the problem have not occurred (yet). So I state the problem, just to see if it's already known and if there's a consensus to do something about it or not. I do not insist. We decide not to do anything for now? That's OK, let's wait for occurrence. I'll offer my help when it will be needed.

The context is important also. First, we are contractors and we bill every change we make to the code. If we find a bug that the clients has not faced yet, it's complicated to inform them and demand money to fix it. It would be like maintaining a ransomware! It's a bad way to do software but the business is OK with that illusion of cost control.

Here is another factor. In our company, people are evaluated for their personal performance, not for the whole team performance. The benefits is that it theoretically push everyone to do its best and be rewarded for it. The problem is that it can easily create unproductive competition between people.

As a new guy in the team I have to figure out who are the leaders. It's useless to get in a fight with them. I propose stuff, and accept that they will be dismissed 80% of the time with a sarcasm. By the way, using sarcasm is lacking factual arguments. It's a good sign!