Build Ubuntu kernel
2017-07-30

Build Ubuntu 16.04 (Xenial) kernel on launchpad

git clone git://kernel.ubuntu.com/ubuntu/ubuntu-xenial.git
cd ubuntu-xenial
fakeroot debian/rules clean
debuild -S

# upload source package to launchpad
dput <Your ppa> ../linux_4.4.0-87.110_source.changes

Build Ubuntu 16.04 (Xenial) HWE kernel on launchpad

git clone git://kernel.ubuntu.com/ubuntu/ubuntu-xenial.git
cd ubuntu-xenial
git checkout origin/hwe -b hwe
fakeroot debian/rules clean
debuild -S

# upload source package to launchpad
dput <Your ppa> ../linux-hwe_4.10.0-29.33~16.04.1_source.changes

Build Ubuntu unstable kernel on launchpad

git clone git://kernel.ubuntu.com/ubuntu/unstable.git
cd unstable
fakeroot debian/rules clean
debuild -S

# upload source package to launchpad
dput <Your ppa> ../linux_4.13.0-2.3_source.changes

Launchpad API
2017-07-06

Launchpad provide a web service api for developer. The main area I use launchpad api on:

  1. Grab/Edit bug information
  2. Grab/Attach attachment

For full functionality, please refer to online document: https://api.launchpad.net/devel.html

There is a launchpadapi python client/library let you use launchpad web api easily. The key concept is to get the launchpad object. And then manipulate the objects.

I recommended trying lp-shell in lptools packages. With lp-shell utility which provides an interactive python shell, developer could use launchpadlib python library interactively and explore functions and properties of launchpadlib. Example:

b = lp.bugs[1]

print(b.title)
print(b.message_count)

Here is more sample scripts: https://help.launchpad.net/API/Examples

Migrate to Hugo
2017-07-01

Due to some software issues not fixing for long time on logdown.com [2], I decided to finding other blog solutions for hosting my tech blog. After considering several options (Such as Hugo, Ghost, Octopress, Jekyll, Jekyll-Now). The final choice goes to Hugo. Hugo is a popular static website generator which built on top of Golang. The community support is actively. Over 18 thousands stars on Hugo github repository. And Debian/Ubuntu has hugo packages which is very easily for normal user to install.

For migrating to Hugo, I exported all my articles on logdown, which content is served with similar format to Octopress. And then dump all the markdown files into content/post/ folder in newly created hugo repository. Everything went well except two steps were needed.

  1. I used sed script in [1] to change date format, otherwise the hugo will print parsing error.
  2. Remove all the
layout: post

line in the octopress markdown article

In fact I spent some time (more thant I expected) to select the Hugo theme. At begining none of the themes seems fit my need (Easy setup, Large font, Readibility, Clear article information such as date information).

hugo-geo

  • Pro: Great design, Great Readibility.
  • Cons: Lack of date and tag information in article page. Only summary in front page.

hyde-x

  • Pro: Simple. Allow showing full article in index.html.
  • Cons: Lack of hugo tag support.

Finally I decided to use hugo-geo theme which is great. And customized hugo-geo by adding my personal wanted features:

  1. Show date information in article and list pages.
  2. Show tag information in article
  3. Show full article in front page.

Customized hugo-geo: https://github.com/swem/hugo-geo

Will file pull request about 1. feature to upstream later. (Which I think might benefit a lot users.)

[1] https://jaketrent.com/post/conversion-from-octopress-to-hugo/

[2] I can’t login via facebook account. This feature have been broken for nearly half year. Another feature broken is url auto-generate.

Automatic dependent surveillance – broadcast (ADS–B) 是 廣播式自動回報監視(ADS-B)[1]。航機每秒發射一次之航機位置、高度、位置完整性、航機識別、航機24 bit位址、速度及其他資料。只要準備一個簡易的 ADS-B 接收器,就可以收到這些訊息。下面介紹怎麼使用 dump1090 ,來觀察這些航機發送出來的訊息。

無論使用哪種方式啟動,都要先確定一些 driver 不會被自動載入。設定 driver blacklist 後,重開機讓設定生效。

$ cat << EOF > /tmp/blacklist-rtl-sdr.conf
blacklist dvb_usb_rtl28xxu
blacklist e4000
blacklist rtl2832
EOF
$ sudo mv /tmp/blacklist-rtl-sdr.conf /etc/modprobe.d
$ sudo reboot

接上 Realtek 2838 DVB-T usb stick。可以用 lsusb 確認己連接

$ lsusb
Bus 003 Device 014: ID 0bda:2838 Realtek Semiconductor Corp. RTL2838 DVB-T

接下來可以選擇使用 docker 或 snap 來啟動 dump1090

Docker

我的測試環境是 Debian 9.0 Stretch AMD64

使用 docker 來執行 docker hub 上己經包好的 dump1090

docker run --rm --it -p 8080:8080 --device=/dev/bus/usb inodes/docker-x86-rtlsdr-dump1090 --net

連接到 http://localhost:8080 就可以看到接收到的飛機訊號

Snap

我的測試環境是 Ubuntu 16.04 Xenial AMD64

安裝 adsb-box 並連接 snap interface。最後重新啟動服務

sudo snap install --beta adsb-box

sudo snap connect adsb-box:raw-usb
sudo snap connect adsb-box:process-control
sudo snap connect adsb-box:system-observe
sudo snap connect adsb-box:network-observe
sudo systemctl restart snap.adsb-box.dump1090.service
sudo systemctl restart snap.adsb-box.piaware.service

連接到 http://localhost:8080 就可以看到接收到的飛機訊號

[1] 飛航服務總臺 > 設施介紹 > 監視裝備

[2] https://github.com/tsunghanliu/adsb-box.snap

[3] https://github.com/mutability/dump1090

Ubuntu 將在 17.10 切換到 Gnome Desktop。其實我覺得 Ubuntu Unity 的 UI 設計是很不錯的,簡潔又不失功能性。Gnome 3 的預設版型我反倒不這麼喜歡。

關於 Gnome Shell 的設計文件:Projects/GnomeShell/Design - GNOME Wiki!

不過使用者可以透過 gnome extensions 來改造自己的 UI。以下介紹 4 個我個人必裝的 gnome extensions:

Recently I’ve trace how cloud-init is being run. First, I try to find service file in /etc/system/system or /lib/systemd/system. Although I can locate the unit files, I can’t see any systemd target that will bring up these cloud-init services. I did’t see how cloud-init.target is activated by systemd.

Finally, I found that cloud-init project utilize the systemd generator to generate the unit files dynamically. The dynamic unit files is located at /run/system/generator.early/multi-user.target.wants/cloud-init.target. With this unit file, when the multi-user.target is activated by systemd, the cloud-init.target will also be activated. Then the systemd will further activate related cloud-init service in /etc/systemd/system/cloud-init.target.wants/.

Below is the dynamic unit file.

$ ll /run/systemd/generator.early/multi-user.target.wants/cloud-init.target
lrwxrwxrwx 1 root root 37 Jul 14 07:52 /run/systemd/generator.early/multi-user.target.wants/cloud-init.target -> /lib/systemd/system/cloud-init.target

If I want to disable cloud-init, there are several ways to do it. 1. Mask the default cloud-init systemd generator.

ln -sf /dev/null /etc/systemd/system-generator/cloud-init-generator
  1. Add kernel parameter cloud-init=disabled or touch /etc/cloud/cloud-init.disabled. This kernel parameter will be processed by cloud-init systemd generator. See http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/systemd/cloud-init.target

Ref: https://www.freedesktop.org/software/systemd/man/systemd.generator.html

Snapper is a great tool for making snapshot on linux. You can download it on ArchLinux/Debian/OpenSUSE/Ubuntu

Personally I use btrfs as underlying file system. Here are some basic command to snapshot my data.

# install snapper
sudo apt install -y snapper

# Suppose /dev/sda3 is formatted as btrfs file system.
# mount the file system
sudo mount /dev/sda3 /mnt

# create a btrfs subvolme "data"
sudo btrfs subvolume create /mnt/data
# unmount /dev/sda3
sudo umount /mnt

# mount only the subvolume "data", instead of whole file system
sudo mount -o subvol=data /dev/sda3 /data

# create the snapper config for "/data" folder. The config name could be anything, here I choose "data"
sudo snapper -c data create-config /data

# By default, the snapper will make a snapshot per hour, just like Mac OS's Time machine.
# Check the snapshot list
# sudo snapper -c data list 
Type   | #    | Pre # | Date                            | User | Cleanup  | Description | Userdata
-------+------+-------+---------------------------------+------+----------+-------------+---------
single | 1 |       | Tue 16 Aug 2016 09:17:02 AM CST | root | timeline | timeline    |         
single | 2 |       | Tue 16 Aug 2016 10:17:02 AM CST | root | timeline | timeline    |         
single | 3 |       | Tue 16 Aug 2016 11:17:03 AM CST | root | timeline | timeline    |         
single | 4 |       | Tue 16 Aug 2016 12:17:03 PM CST | root | timeline | timeline    |      

# In fact, you can find the underlying snapshot in "data" subvolume
sudo mount /dev/sda3 /mnt
ls -al /mnt/data/.snapshots/
drwxr-x--- 1 root root 230 Aug 16 12:17 ./
drwxr-xr-x 1 root root 340 Aug 15 10:47 ../
drwxr-xr-x 1 root root  32 Aug 16 09:17 1/
drwxr-xr-x 1 root root  32 Aug 16 10:17 2/
drwxr-xr-x 1 root root  32 Aug 16 11:17 3/
drwxr-xr-x 1 root root  32 Aug 16 12:17 4/

If your root file system is also a btrfs subvolume. That would be even better. You can snapshot your whole system!

sudo snapper -c root create-config /

Below I give some simple command to run a ubuntu:16.04 lxd container. LXD container is fast, very efficient, very low-footprint virtual machine. I recommend people who didn’t need low-level system control (such as disk, loopback device) give it a try. I think this kind of lightweight virtual machine would benefit people who study machine learning or similar scientific computing.

# check current status of lxd. If you install lxd the first time, it should containing no virtual machines.
lxc list

# if my-ubuntu virtual machine not exist yet.
lxc launch ubuntu:16.04 my-ubuntu
# if my-ubuntu virtual machine already exist.
lxc start my-ubuntu

# check status of my-ubuntu virtual machine
lxc list

# login to root account
lxc exec first -- /bin/bash
# or login to ubuntu account
lxc exec first -- /bin/su - ubuntu

# stop the virtual machine
lxc stop my-ubuntu

# you can even copy virtual machine
lxc copy my-ubuntu my-ubuntu2
# or rename it
lxc mv my-ubuntu stanley-ubuntu
lxc mv stanley-ubuntu my-ubuntu

# delete my-ubuntu virtual machine
lxc delete my-ubuntu

# check my-ubuntu is deleted
lxc list

Varies images is provided by Canonical in http://images.linuxcontainers.org/

Note that lxd has some limitaions. Such as you can’t - setup loopback device - mount fuse file system - have bridged network. - load/remove kernel module … (But if you choose to run a priviledged lxd virtual machine, some limitations might not exist anymore. But I haven’t dig into it currently.)

Note that this article introduces how to build Debian/Ubuntu LiveCD iso image from scratch. With this method, you have great flexibility to customize the LiveCD.

However, if you only want to customize the LiveCD a little bit. Such as adding some packages for installation, or changing the preseed configuration, you can refer to DebianCustomCD, Simple-CDD and LiveCDCustomization

Debian Jessie

# within Debian jessie
git clone git://anonscm.debian.org/git/debian-live/live-build.git
cd live-build
git checkout debian/4.0.5-1
dpkg-buildpackage -b -uc -us
cd ..
sudo dpkg -i live-build_4.0.5-1_all.deb

sudo apt-get install -y live-images
cp -r /usr/share/live/images/standard .
cd standard/
lb config --source false # optional, would be faster if not build source
sudo lb build
# here we got live-image-amd64.hybrid.iso, about 417MB
# refer to http://cdimage.debian.org/mirror/cdimage/archive/8.6.0-live/amd64/iso-hybrid/debian-live-8.6.0-amd64-standard.iso.log for more information

Ubuntu Xenial 16.04

Automatically build with ubuntu-defaults-builder

sudo apt install -y ubuntu-defaults-builder
sudo apt install -y syslinux-utils
ubuntu-defaults-template myconfig
cd myconfig/
dpkg-buildpackage
cd ../
ubuntu-defaults-image --package myconfig_0.1_all.deb --components main,restricted,universe,multiverse

# here we got binary.hybrid.iso and livecd.ubuntu.iso

build with ubuntu-cdimage

TODO:

Manually build with live-build (Not working right now, might need hooks to allow booting)

# First, set up the build tools and workspace.
# The scripts require that you work in /build

sudo apt-get install -y genisoimage syslinux-utils # tools for generate ISO image
sudo apt-get install -y memtest86+ syslinux syslinux-themes-ubuntu-xenial gfxboot-theme-ubuntu

sudo -i
apt-get install -y livecd-rootfs
mkdir -p /build/chroot
cd /build
cp -a /usr/share/livecd-rootfs/live-build/auto .

# All the hard work is done with live-build (lb command)
# and we have to configure it with environment variables

export SUITE=xenial
export ARCH=amd64
export PROJECT=ubuntu
export MIRROR=http://archive.ubuntu.com/ubuntu/
export BINARYFORMAT=iso-hybrid
export LB_SYSLINUX_THEME=ubuntu-xenial

# Now we can have live-build set up the workspace

lb config --initramfs-compression=gzip 

# And finally, start the build

lb build

Ref: [1] https://code.launchpad.net/~semiosis/livecd-rootfs/fix-for-1565985/+merge/298305/comments/768010 [2] Ubuntu live filesystem builds moved to live-build [3] LocalizedCDImageTools - Ubuntu Wiki [4] ISO with l10 preloaded for LoCo + UEFI [5] UbuntuStudio/Seeds - Ubuntu Wiki [6] Building large-scale Ubuntu derivatives using seeds - Google Docs [7] ReleaseTeam/CDImageSetup - Ubuntu Wiki

I highly recommened pudb, which is a full-screen, console-based visual debugger for Python. The user-interface is nicely designed for python developers.

Here is an example debugging a simple python script: Introduction to the PuDB Python Debugging Tool

Also, another great tutorial is from Jordi Gutiérrez Hermoso. Montreal, QC, September 14, 2015 - Jordi Gutiérrez Hermoso presents PuDB, a full-screen, console-based visual debugger for Python. An interesting comment from the speaker: > Everyone should use a debugger > … > > But If pdb is the only debugger you’ve every seen, I won’t blame you not using a debugger > > … > because the debugger is ugly, not because the source code is ugly

To install pudb on Debian/Ubuntu, you can use apt-get to install the package

# for python 2
sudo apt-get -y python-pudb
# for python 3
sudo apt-get -y python3-pudb

# pudb is even better when debugging with ipython
# so that's install ipython
sudo apt-get install ipython