Openwrt on Philly Mesh

Building DIY Community Mesh Networks (2600 Article)

Mike Dank (Famicoman) — Wed, 21 Dec 2016 23:08:09 +0000

Now that the article has been printed in 2600 magazine, Volume 33, Issue 3 (2016-10-10), I’m able to republish it on the web. The article below is my submission to 2600 with some slight formatting changes for hyperlinks.

Building DIY Community Mesh Networks

By Mike Dank
Famicoman@gmail.com

Today, we are faced with issues regarding our access to the Internet, as well as our freedoms on it. As governmental bodies fight to gain more control and influence over the flow of our information, some choose to look for alternatives to the traditional Internet and build their own networks as they see fit. These community networks can pop up in dense urban areas, remote locations with limited Internet access, and everywhere in between.Whether you are politically fueled by issues of net neutrality, privacy, and censorship, fed up with an oligarchy of Internet service providers, or just like tinkering with hardware, a wireless mesh network (or “meshnet”) can be an invaluable project to work on. Numerous groups and organizations have popped up all over the world, creating robust mesh networks and refining the technologies that make them possible. While the overall task of building a wireless mesh network for your community may seem daunting, it is easy to get started and scale up as needed.

What Are Mesh Networks?

Think about your existing home network. Most people have a centralized router with several devices hooked up to it. Each device communicates directly with the central router and relies on it to relay traffic to and from other devices. This is called a hub/spoke topology, and you’ll notice that it has a single point of failure. With a mesh topology, many different routers (referred to as nodes) relay traffic to one another on the path to the target machine. Nodes in this network can be set up ad-hoc; if one node goes down, traffic can easily be rerouted to another node. If new nodes come online, they can be seamlessly integrated into the network. In the wireless space, distant users can be connected together with the help of directional antennas and share network access. As more nodes join a network, service only improves as various gaps are filled in and connections are made more redundant. Ultimately, a network is created that is both decentralized and distributed. There is no single point of failure, making it difficult to shut down.

When creating mesh networks, we are mostly concerned with how devices are routing to and linking with one another. This means that most services you are used to running like HTTP or IRC daemons should be able to operate without a hitch. Additionally, you are presented with the choice of whether or not to create a darknet (completely separated from the Internet) or host exit nodes to allow your traffic out of the mesh.

Existing Community Mesh Networking Projects

One of the most well-known grassroots community mesh networks is Freifunk, based out of Germany, encompassing over 150 local communities with over 25,000 access points. Guifi.net based in Spain, boasts over 27,000 nodes spanning over 36,000 km. In North America we see projects like Hyperboria which connect smaller mesh networking communities together such as Seattle Meshnet, NYC Mesh, andToronto Mesh. We also see standalone projects like PittMesh in Pittsburgh, WasabiNet in St. Louis, and People’s Open Network in Oakland, California.

While each of these mesh networks may run different software and have a different base of users, they all serve an important purpose within their communities. Additionally, many of these networks consistently give back to the greater mesh networking community and choose to share information about their hardware configurations, software stacks, and infrastructure. This only benefits those who want to start their own networks or improve existing ones.

Picking Your Hardware & OS

When I was first starting out with Philly Mesh, I was faced with the issue of acquiring hardware on a shoestring budget. Many will tell you that the best hardware is low-power computers with dedicated wireless cards. This however can incur a cost of several hundred dollars per node. Alternatively, many groups make use of SOHO routers purchased off-the-shelf, flashed with custom firmware. The most popular firmware used here is OpenWRT, an open source alternative that supports a large majority of consumer routers. If you have a relatively modern router in your house, there is a good chance it is already supported (if you are buying specifically for meshing, consider consulting OpenWRT’s wiki for compatibility. Based on Linux, OpenWRT really shines with its packaging system, allowing you to easily install and configure packages of networking software across several routers regardless of most hardware differences between nodes. With only a few commands, you can have mesh packages installed and ready for production.

Other groups are turning towards credit-card-sized computers like the BeagleBone Black and Raspberry Pi, using multiple USB WiFi dongles to perform over-the-air communication. Here, we have many more options for an operating system as many prefer to use a flavor of Linux or BSD, though most of these platforms also have OpenWRT support.

There are no specific wrong answers here when choosing your hardware. Some platforms may be better suited to different scenarios. For the sake of getting started, spec’ing out some inexpensive routers (aim for something with at least two radios, 8MB of flash) or repurposing some Raspberry Pis is perfectly adequate and will help you learn the fundamental concepts of mesh networking as well develop a working prototype that can be upgraded or expanded as needed (hooray for portable configurations). Make sure you consider options like indoor vs outdoor use, 2.4 GHz vs. 5 GHz band, etc.

Meshing Software

You have OpenWRT or another operating system installed, but how can you mesh your router with others wirelessly? Now, you have to pick out some software that will allow you to facilitate a mesh network. The first packages that you need to look at are for what is called the data link layer of the OSI model of computer networking (or OSI layer 2). Software here establishes the protocol that controls how your packets get transferred from node A to node B. Common software in this space is batman-adv (not to be confused with the layer 3 B.A.T.M.A.N. daemon), and open80211s, which are available for most operating systems. Each of these pieces of software have their own strengths and weaknesses; it might be best to install each package on a pair of routers and see which one works best for you. There is currently a lot of praise for batman-adv as it has been integrated into the mainline Linux tree and was developed by Freifunk to use within their own mesh network.

Revisiting the OSI model again, you will also need some software to work at the network layer (OSI layer 3). This will control your IP routing, allowing for each node to compute where to send traffic next on its forwarding path to the final destination on the network. There are many software packages here such as OLSR (Optimized Link State Routing), B.A.T.M.A.N (Better Approach To Mobile Adhoc Networking), Babel, BMX6, and CJDNS (Caleb James Delisle’s Networking Suite). Each of these addresses the task in its own way, making use of a proactive, reactive, or hybrid approach to determine routing. B.A.T.M.A.N. and OLSR are popular here, both developed by Freifunk. Though B.A.T.M.A.N. was designed as a replacement for OLSR, each is actively used and OLSR is highly utilized in the Commotion mesh networking firmware (a router firmware based off of OpenWRT).

For my needs, I settled on CJDNS which boasts IPv6 addressing, secure communications, and some flexibility in auto-peering with local nodes. Additionally, CJDNS is agnostic to how its host connects to peers. It will work whether you want to connect to another access point over batman-adv, or even tunnel over the existing Internet (similar to Tor or a VPN)! This is useful for mesh networks starting out that may have nodes too distant to connect wirelessly until more nodes are set up in-between. This gives you a chance to lay infrastructure sooner rather than later, and simply swap-out for wireless linking when possible. You also get the interesting ability to link multiple meshnets together that may not be geographically close.

Putting It Together

At this point, you should have at least one node (though you will probably want two for testing) running the software stack that you have settled on. With wireless communications, you can generally say that the higher you place the antenna, the better. Many community mesh groups try to establish nodes on top of buildings with roof access, making use of both directional antennas (to connect to distant nodes within the line of sight) as well as omnidirectional antennas to connect to nearby nodes and/or peers. By arranging several distant nodes to connect to one another via line of sight, you can establish a networking backbone for your meshnet that other nodes in the city can easily connect to and branch off of.

Gathering Interest

Mesh networks can only grow so much when you are working by yourself. At some point, you are going to need help finding homes for more nodes and expanding the network. You can easily start with friends and family – see if they are willing to host a node (they probably wouldn’t even notice it after a while). Otherwise, you will want to meet with like-minded people who can help configure hardware and software, or plan out the infrastructure. You can start small online by setting up a website with a mission statement and making a post or two on Reddit (/r/darknetplan in particular) or Twitter. Do you have hackerspaces in your area? Linux or amateur radio groups? A 2600 meeting you frequent? All of these are great resources to meet people face-to-face and grow your network one node at a time.

Conclusion

Starting a mesh network is easier than many think, and is an incredible way to learn about networking, Linux, micro platforms, embedded systems, and wireless communication. With only a few off-the-shelf devices, one can get their own working network set up and scale it to accommodate more users. Community-run mesh networks not only aid in helping those fed up with or persecuted by traditional network providers, but also those who want to construct, experiment, and tinker. With mesh networks, we can build our own future of communication and free the network for everyone.

CJDNS on OpenWRT – Part 3: Installing & Configuring CJDNS

Mike Dank (Famicoman) — Thu, 21 Apr 2016 21:49:08 +0000

Now that we have OpenWRT installed and ensured that we have enough space to experiment and install packages, we can proceed to install and configure cjdns.

I have opted to install a GUI package to allow for easier configuration (though I also wanted to see what it had to offer over editing configuration files). The package used here is luci-app-cjdns, relying on the LuCI interface that comes default in most OpenWRT images. If you want to install cjdns without the GUI or do not use LuCI, you can install the regular cjdns package. Note: The standard cjdns package was left out of OpenWRT 15.05.1, but should be in the older 15.05 image. The luci-app-cjdns package should be available in both versions, so you won’t have any issue with the remainder of this guide.

Now we are ready to install cjdns for LuCI. SSH into the access point and run the following command to update and install luci-app-cjdns.

opkg update && opkg install luci-app-cjdns

After this finishes, leave the SSH session open and then load up the OpenWRT web console in a browser and log in. By default, this interface can be reached via http://192.168.1.1. Now that we’re in the console, select cjdns from the Services dropdown on the top menu. An Overview page for cjdns will load (and look rather empty). Now, click the Peers sub-tab link near the top of this page.

Now, we can enter in the peering information for any number of peers to connect to. You will likely want to populate the Authorized Passwords and Outgoing UDP Peers sections as I have below.

CJDNS Peers Tab

When finished, press the Save & Apply button to commit any changes and restart cjdns. These steps can be repeated to add as many peers as needed.

Now, navigate back to the Overview page by clicking on the Overview sub-tab link.

After loading, we should now have connection information about the configured peers as shown below.

CJDNS Overview Page

That’s all there is to it! Back in our SSH session, we can try pinging a machine on Hyperboria to confirm a connection:

ping6 h.peer0.famicoman.com
PING h.peer0.famicoman.com (fc9f:990d:2b0f:75ad:8783:5d59:7c84:520b): 56 data bytes
64 bytes from fc9f:990d:2b0f:75ad:8783:5d59:7c84:520b: seq=0 ttl=42 time=4072.631 ms
64 bytes from fc9f:990d:2b0f:75ad:8783:5d59:7c84:520b: seq=1 ttl=42 time=3800.924 ms
64 bytes from fc9f:990d:2b0f:75ad:8783:5d59:7c84:520b: seq=2 ttl=42 time=4594.193 ms
64 bytes from fc9f:990d:2b0f:75ad:8783:5d59:7c84:520b: seq=3 ttl=42 time=4329.846 ms
^C
--- h.peer0.famicoman.com ping statistics ---
9 packets transmitted, 4 packets received, 55% packet loss
round-trip min/avg/max = 3800.924/4199.398/4594.193 ms

If all went as expected, you now have cjdns running on your OpenWRT router! This can be expanded in the future by copying an OpenWRT configuration onto several routers, and then linking them together wirelessly.

CJDNS on OpenWRT – Part 2: Configuring Extroot for More Storage

Mike Dank (Famicoman) — Mon, 11 Apr 2016 21:46:56 +0000

If you have any low-memory OpenWRT device (4MB of flash) you will probably fill up any free space quickly after the initial OpenWRT install and need more room to grow. Luckily, you can transfer your root file system to a flash drive and boot off of it as long as your access point has a USB port.

If you are following along with our Western Digital N600, you probably don’t need to do this. The N600 comes equipped with 12MB of built-in flash, more than enough to accommodate the software packages we will install in the future. If you have less than this or want to have a nice learning exercise, read on!

You are going to need a Linux machine and a flash drive. The flash drive size shouldn’t matter too much. A lot of people run OpenWRT off of 8MB of internal flash, so any small drive should have plenty of room.

On your Linux machine, plug in the flash drive (mine is a ~10 year old 64MB), and run dmesg to get the kernel message buffer.

dmesg

You should get a lot of output, but importantly at the end, we should see our flash drive being recognized:

[26913.782811] usb 1-1.4: new high-speed USB device number 4 using dwc_otg
[26913.883754] usb 1-1.4: New USB device found, idVendor=0457, idProduct=0151[26913.883779] usb 1-1.4: New USB device strings: Mfr=0, Product=2, SerialNumber=3
[26913.883796] usb 1-1.4: Product: USB Mass Storage Device
[26913.883812] usb 1-1.4: SerialNumber: 00000000004FDE
[26913.884913] usb-storage 1-1.4:1.0: USB Mass Storage device detected
[26913.887282] usb-storage 1-1.4:1.0: Quirks match for vid 0457 pid 0151: 80
[26913.887450] scsi host0: usb-storage 1-1.4:1.0
[26914.884185] scsi 0:0:0:0: Direct-Access Staples 0.00 PQ: 0 ANSI: 2
[26914.886688] sd 0:0:0:0: [sda] 124000 512-byte logical blocks: (63.4 MB/60.5 MiB)
[26914.887210] sd 0:0:0:0: [sda] Write Protect is off
[26914.887235] sd 0:0:0:0: [sda] Mode Sense: 00 00 00 00
[26914.887748] sd 0:0:0:0: [sda] Asking for cache data failed
[26914.887771] sd 0:0:0:0: [sda] Assuming drive cache: write through
[26914.916959] sda: sda1
[26914.920057] sd 0:0:0:0: [sda] Attached SCSI removable disk
[26914.922645] sd 0:0:0:0: Attached scsi generic sg0 type 0

We see that our physical device is sda, with one partition sda1. Your drive/partition may be labeled differently depending on how many drives you have installed or plugged into your machine, and how many partitions your flash drive has. We can verify we are looking at our flash drive by listing via fdisk.

fdisk -l /dev/sda

You will get a lot of informative output about the device:

Disk /dev/sda: 63 MB, 63488000 bytes
16 heads, 32 sectors/track, 242 cylinders, total 124000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x91f72d24
Device Boot Start End Blocks Id System
/dev/sda1 * 32 123999 61984 b W95 FAT32

Now we will go ahead and format the drive as ext4. First however, we need to create a partition by running the fdisk command on the drive (without the -l option).

fdisk /dev/sda

This is an interactive utility, so when prompted, enter d to delete the current partition on the drive. Then enter n< for a new partition (taking the defaults by pressing the return key). Finally, enter w to apply the changes to the disk and exit.

Now, we can make a file system on the partition we just created, formatting it as ext4:

mkfs.ext4 /dev/sda1

Afterwards we can list with fdisk again to see our changes:

fdisk -l /dev/sda
Disk /dev/sda: 63 MB, 63488000 bytes
3 heads, 32 sectors/track, 1291 cylinders, total 124000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x91f72d24
Device Boot Start End Blocks Id System
/dev/sda1 2048 123999 60976 83 Linux

You can now remove your USB drive from your Linux machine and plug it into your OpenWRT device.

Next, we ssh into our OpenWRT device and login as root. We need to install a few utilities with opkg before we can switch over the root filesystem.

opkg update && opkg install block-mount kmod-fs-ext4 kmod-usb-storage fdisk nano

Now we will run fdisk to see that our drive is recognized:

root@OpenWrt:~# fdisk -l
Disk /dev/mtdblock0: 256 KiB, 262144 bytes, 512 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/mtdblock1: 64 KiB, 65536 bytes, 128 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/mtdblock2: 64 KiB, 65536 bytes, 128 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/mtdblock3: 64 KiB, 65536 bytes, 128 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/mtdblock4: 15.5 MiB, 16252928 bytes, 31744 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/mtdblock5: 1.3 MiB, 1310720 bytes, 2560 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/mtdblock6: 14.3 MiB, 14942208 bytes, 29184 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/mtdblock7: 12.1 MiB, 12648448 bytes, 24704 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/mtdblock8: 64 KiB, 65536 bytes, 128 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/sda: 60.6 MiB, 63488000 bytes, 124000 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x91f72d24
Device Boot Start End Sectors Size Id Type
/dev/sda1 2048 123999 121952 59.6M 83 Linux

My drive registers as /dev/sda1, and this is used through the commands below. Make sure you take note of yours and make the necessary changes when running additional commands.

Next, we will copy our filesystem over and set the device to boot from the flash drive. If you are squeamish about making this change via a terminal, scroll down and view the sources referenced at the end of this tutorial. There is a way to perform this configuration using OpenWRT’s default graphical interface: LuCI. If you prefer to issue commands over ssh, read on.

Now, we will actually mount the drive, and copy the root filesystem to it:

mkdir /mnt/sda1
mount /dev/sda1 /mnt/sda1
mkdir -p /tmp/cproot
mount --bind / /tmp/cproot
tar -C /tmp/cproot -cvf - . | tar -C /mnt/sda1 -xf -
umount /tmp/cproot

After we have moved the filesystem over, we can modify the fstab with our editor of choice.

nano /etc/config/fstab

Paste the following at the top the file and save it. This will use the flash drive as the primary filesystem and preserve this configuration even after reboot.

config mount
option device '/dev/sda1'
option target '/'
option fstype 'ext4'
option enabled '1'

Finally, we reboot the device by issuing the reboot command:

reboot

After the system comes back up, ssh into it once more and run the df command.

df -h

In the output, pay attention to the Size column to make sure it matches up closely to your flash drive’s capacity to verify that you are actually running off of the flash drive.

Filesystem Size Used Available Use% Mounted on
rootfs 53.7M 9.7M 39.8M 20% /
/dev/root 2.3M 2.3M 0 100% /rom
tmpfs 61.6M 628.0K 61.0M 1% /tmp
/dev/sda1 53.7M 9.7M 39.8M 20% /
tmpfs 512.0K 0 512.0K 0% /dev

That’s all there is to it! Now, you are booting directly off of the flash drive and have more room to install packages. Additionally, you may want to save an image of your drive for future use. You never know when you will be trying something out and need to restore from a backup!

Sources

CJDNS on OpenWRT – Part 1: Installing OpenWRT on the WD N600

Mike Dank (Famicoman) — Thu, 31 Mar 2016 21:42:38 +0000

I was lucky enough to snag a Western Digital N600 router recently for $10 via Woot and have been working through the process of getting it configured with OpenWRT and cjdns.

For $10, I didn’t think I was getting a whole lot, but these devices sport a popular Atheros chipset and are perfectly compatible with OpenWRT’s latest version (Chaos Calmer 15.05 at the time of writing). For the uninitiated, OpenWRT is an alternative firmware for routers that allows for an advanced set of features and more customization.

My N600, happily chugging away.

The first (and sometimes daunting) task in this process is to flash the firmware on to the device, but this is easy to accomplish with the help of the OpenWRT Wiki page for the N600.

One issue I’ve found is that the page states that the web updater doesn’t work on most N600 devices and that it is preferable to use telnet. Being the console junkie I am, I tried the telnet method first but had no way to configure or enable it! I found I couldn’t use telnet but ultimately was able to flash via the web interface.

Adapted from the wiki, here are the steps I took to flash my device. Any additions/modifications by me are in bold:

0) Turn on and configure the device. I couldn’t do anything until i completed the initial setup.

Download the file openwrt-ar71xx-generic-mynet-n600-squashfs-factory.bin. I pulled it down from https://downloads.openwrt.org/chaos_calmer/15.05/ar71xx/generic/openwrt-15.05-ar71xx-generic-mynet-n600-squashfs-factory.bin

Configure your computers IP address to 192.168.1.10 and connect to a LAN port in the router.

Turn the router off.

Using a paperclip, press and hold the reset button on the bottom of the router and turn it on. Hold the reset button for at least 15 seconds. Wait until the power light on the front is slowly flashing on and off.

On your computer, visit http://192.168.1.1 NOTE: You will not be able to ping this address.

Upload the file openwrt-ar71xx-generic-mynet-n600-squashfs-factory.bin as downloaded earlier.

The router will now flash OpenWRT. This will take a couple of minutes to achieve. You can ping 192.168.1.1 and watch for ping replies to see when your router has rebooted into OpenWRT.

That’s all there is to it. After OpenWRT boots for the first time, you’ll be able to configure it to work on your local network. At most,I suggest setting a root password!

This process is pretty adaptable to other hardware, and the OpenWRT wiki is invaluable when it comes to specific steps to flash OpenWRT on any device you may have around. Don’t feel like you have to get an N600 because I did, a lot of hardware is supported (I would recommend something with two radios and 8MB of flash for maximum hackery)!

This tutorial is just the first in a series where we will get cjdns configured on your OpenWRT router. The more meshing, the merrier!