Cover image of BSD Now
(61)
Education
How To
News
Tech News

BSD Now

Updated 6 days ago

Education
How To
News
Tech News
Read more

Created by three guys who love BSD, we cover the latest news and have an extensive series of tutorials, as well as interviews with various people from all areas of the BSD community. It also serves as a platform for support and questions. We love and advocate FreeBSD, OpenBSD, NetBSD, DragonFlyBSD and TrueOS. Our show aims to be helpful and informative for new users that want to learn about them, but still be entertaining for the people who are already pros.The show airs on Wednesdays at 2:00PM (US Eastern time) and the edited version is usually up the following day.

Read more

Created by three guys who love BSD, we cover the latest news and have an extensive series of tutorials, as well as interviews with various people from all areas of the BSD community. It also serves as a platform for support and questions. We love and advocate FreeBSD, OpenBSD, NetBSD, DragonFlyBSD and TrueOS. Our show aims to be helpful and informative for new users that want to learn about them, but still be entertaining for the people who are already pros.The show airs on Wednesdays at 2:00PM (US Eastern time) and the edited version is usually up the following day.

iTunes Ratings

61 Ratings
Average Ratings
56
3
1
1
0

Great for keeping up wth BSDs and Unix

By brysonholland - Mar 19 2019
Read more
They don't just cover the BSDs, but the wider Unix world as well. Great topics, technical analysis, and interviews. One of the best Unix/BSD podcasts currently out there.

Best beginner-accessible BSD resource anywhere!

By ProCoreyJ - Dec 12 2016
Read more
This is the best source ofBSD info available to beginners but also doesn't hesitate to dive into the interesting details holding it all together. I have found this podcast to be an essential resource for getting and staying current with tools, techniques, and community to make BSD work for me. Thanks for putting this on! Additionally, I think this podcast is probably the best marketing arm for the community. Keep it available!

iTunes Ratings

61 Ratings
Average Ratings
56
3
1
1
0

Great for keeping up wth BSDs and Unix

By brysonholland - Mar 19 2019
Read more
They don't just cover the BSDs, but the wider Unix world as well. Great topics, technical analysis, and interviews. One of the best Unix/BSD podcasts currently out there.

Best beginner-accessible BSD resource anywhere!

By ProCoreyJ - Dec 12 2016
Read more
This is the best source ofBSD info available to beginners but also doesn't hesitate to dive into the interesting details holding it all together. I have found this podcast to be an essential resource for getting and staying current with tools, techniques, and community to make BSD work for me. Thanks for putting this on! Additionally, I think this podcast is probably the best marketing arm for the community. Keep it available!
Cover image of BSD Now

BSD Now

Latest release on Jan 16, 2020

Read more

Created by three guys who love BSD, we cover the latest news and have an extensive series of tutorials, as well as interviews with various people from all areas of the BSD community. It also serves as a platform for support and questions. We love and advocate FreeBSD, OpenBSD, NetBSD, DragonFlyBSD and TrueOS. Our show aims to be helpful and informative for new users that want to learn about them, but still be entertaining for the people who are already pros.The show airs on Wednesdays at 2:00PM (US Eastern time) and the edited version is usually up the following day.

Rank #1: Episode 239: The Return To ptrace | BSD Now 239

Podcast cover
Read more

OpenBSD firewalling Windows 10, NetBSD’s return to ptrace, TCP Alternative Backoff, the BSD Poetic license, and AsiaBSDcon 2018 videos available.

RSS Feeds:

MP3 Feed | iTunes Feed | HD Vid Feed | HD Torrent Feed

Become a supporter on Patreon:

- Show Notes: -

Headlines

Preventing Windows 10 and untrusted software from having full access to the internet using OpenBSD

Whilst setting up one of my development laptops to port some software to Windows I noticed Windows 10 doing crazy things like installing or updating apps and games by default after initial setup. The one I noticed in particular was Candy Crush Soda Saga which for those who don't know of it is some cheesy little puzzle game originally for consumer devices. I honestly did not want software like this near to a development machine. It has also been reported that Windows 10 now also updates core system software without notifying the user. Surely this destroys any vaguely deterministic behaviour, in my opinion making Windows 10 by default almost useless for development testbeds.

Deciding instead to start from scratch but this time to set the inbuilt Windows Firewall to be very restrictive and only allow a few select programs to communicate. In this case all I really needed to be online was Firefox, Subversion and Putty. To my amusement (and astonishment) I found out that the Windows firewall could be modified to give access very easily by programs during installation (usually because this task needs to be done with admin privileges). It also seems that Windows store Apps can change the windows firewall settings at any point. One way to get around this issue could be to install a 3rd party firewall that most software will not have knowledge about and thus not attempt to break through. However the only decent firewall I have used was Sygate Pro which unfortunately is no longer supported by recent operating systems. The last supported versions was 2003, XP and 2000. In short, I avoid 3rd party firewalls.

Instead I decided to trap Windows 10 (and all of it's rogue updaters) behind a virtual machine running OpenBSD. This effectively provided me with a full blown firewall appliance. From here I could then allow specific software I trusted through the firewall (via a proxy) in a safe, controlled and deterministic manner. For other interested developers (and security conscious users) and for my own reference, I have listed the steps taken here:

  • 1) First and foremost disable the Windows DHCP service - this is so no IP can be obtained on any interface. This effectively stops any communication with any network on the host system. This can be done by running services.msc with admin privileges and stopping and disabling the service called DHCP Client.

  • 2) Install or enable your favorite virtualization software - I have tested this with both VirtualBox and Hyper-V. Note that on non-server versions of Windows, in order to get Hyper-V working, your processor also needs to support SLAT which is daft so to avoid faffing about, I recommend using VirtualBox to get round this seemingly arbitrary restriction.

  • 3) Install OpenBSD on the VM - Note, if you decide to use Hyper-V, its hardware support isn't 100% perfect to run OpenBSD and you will need to disable a couple of things in the kernel. At the initial boot prompt, run the following commands.


config -e -o /bsd /bsd
disable acpi
disable mpbios

  • 4) Add a host only virtual adapter to the VM - This is the one which we are going to connect through the VM with. Look at the IP that VirtualBox assigns this in network manager on the host machine. Mine was [b]192.168.56.1[/b]. Set up the adapter in the OpenBSD VM to have a static address on the same subnet. For example [b]192.168.56.2[/b]. If you are using Hyper-V and OpenBSD, make sure you add a "Legacy Interface" because no guest additions are available. Then set up a virtual switch which is host only.

  • 5) Add a bridged adapter to the VM - then assign it to whichever interface you wanted to connect to the external network with. Note that if using Wireless, set the bridged adapters MAC address to the same as your physical device or the access point will reject it. This is not needed (or possible) on Hyper-V because the actual device is "shared" rather than bridged so the same MAC address is used. Again, if you use Hyper-V, then add another virtual switch and attach it to your chosen external interface. VMs in Hyper-V "share" an adapter within a virtual switch and there is the option to also disable the hosts ability to use this interface at the same time which is fine for an additional level of security if those pesky rogue apps and updaters can also enable / disable DHCP service one day which wouldn't be too surprising.

  • 6) Connect to your network in the host OS - In case of Wireless, select the correct network from the list and type in a password if needed. Windows will probably say "no internet available", it also does not assign an IP address which is fine.

  • 7) Install the Squid proxy package on the OpenBSD guest and enable the daemon

```

pkg_add squid

echo 'squid_flags=""' >> /etc/rc.conf.local

/etc/rc.d/squid start

```

We will use this service for a limited selection of "safe and trusted" programs to connect to the outside world from within the Windows 10 host. You can also use putty on the host to connect to the VM via SSH and create a SOCKS proxy which software like Firefox can also use to connect externally.

  • 8) Configure the software you want to be able to access the external network with

    • Firefox - go to the connection settings and specify the VMs IP address for the proxy.
    • Subversion - modify the %HOME%\AppData\Roaming\Subversion\servers file and change the HTTP proxy field to the VMs IP. This is important to communicate with GitHub via https:// (Yes, GitHub also supports Subversion). For svn:// addresses you can use Putty to port forward.
    • Chromium/Chrome - unfortunately uses the global Windows proxy settings which defeats much of the purpose of this exercise if we were going to allow all of Windows access to the internet via the proxy. It would become mayhem again. However we can still use Putty to create a SOCKS proxy and then launch the browser with the following flags:


--proxy-server="socks5://:"
--host-resolver-rules="MAP * 0.0.0.0 , EXCLUDE "

  • 9) Congratulations, you are now done - Admittedly this process can be a bit fiddly to set up but it completely prevents Windows 10 from making a complete mess. This solution is probably also useful for those who like privacy or don't like the idea of their software "phoning home". Hope you find this useful and if you have any issues, please feel free to leave questions in the comments.

LLDB restoration and return to ptrace(2)

I've managed to unbreak the LLDB debugger as much as possible with the current kernel and hit problems with ptrace(2) that are causing issues with further work on proper NetBSD support. Meanwhile, I've upstreamed all the planned NetBSD patches to sanitizers and helped other BSDs to gain better or initial support.

  • LLDB

Since the last time I worked on LLDB, we have introduced many changes to the kernel interfaces (most notably related to signals) that apparently fixed some bugs in Go and introduced regressions in ptrace(2). Part of the regressions were noted by the existing ATF tests. However, the breakage was only marked as a new problem to resolve. For completeness, the ptrace(2) code was also cleaned up by Christos Zoulas, and we fixed some bugs with compat32.

I've fixed a crash in *NetBSD::Factory::Launch(), triggered on startup of the lldb-server application.

Here is the commit message:

```
We cannot call process_up->SetState() inside
the NativeProcessNetBSD::Factory::Launch
function because it triggers a NULL pointer
deference.

The generic code for launching a process in:
GDBRemoteCommunicationServerLLGS::LaunchProcess
sets the mdebuggedprocessup pointer after
a successful call to m
processfactory.Launch().
If we attempt to call process
up->SetState()
inside a platform specific Launch function we
end up dereferencing a NULL pointer in
NativeProcessProtocol::GetCurrentThreadID().

Use the proper call processup->SetState(,false)
that sets notify
delegates to false.
```

  • Sanitizers

I suspended development of new features in sanitizers last month, but I was still in the process of upstreaming of local patches. This process was time-consuming as it required rebasing patches, adding dedicated tests, and addressing all other requests and comments from the upstream developers.

I'm not counting hot fixes, as some changes were triggering build or test issues on !NetBSD hosts. Thankfully all these issues were addressed quickly. The final result is a reduction of local delta size of almost 1MB to less than 100KB (1205 lines of diff). The remaining patches are rescheduled for later, mostly because they depend on extra work with cross-OS tests and prior integration of sanitizers with the basesystem distribution. I didn't want to put extra work here in the current state of affairs and, I've registered as a mentor for Google Summer of Code for the NetBSD Foundation and prepared Software Quality improvement tasks in order to outsource part of the labour.

  • Userland changes

I've also improved documentation for some of the features of NetBSD, described in man-pages. These pieces of information were sometimes wrong or incomplete, and this makes covering the NetBSD system with features such as sanitizers harder as there is a mismatch between the actual code and the documented code.

Some pieces of software also require better namespacing support, these days mostly for the POSIX standard. I've fixed few low-hanging fruits there and requested pullups to NetBSD-8(BETA).

I thank the developers for improving the landed code in order to ship the best solutions for users.

  • BSD collaboration in LLVM

A One-man-show in human activity is usually less fun and productive than collaboration in a team. This is also true in software development. Last month I was helping as a reviewer to port LLVM features to FreeBSD and when possible to OpenBSD. This included MSan/FreeBSD, libFuzzer/FreeBSD, XRay/FreeBSD and UBSan/OpenBSD.

I've landed most of the submitted and reviewed code to the mainstream LLVM tree.

Part of the code also verified the correctness of NetBSD routes in the existing porting efforts and showed new options for improvement. This is the reason why I've landed preliminary XRay/NetBSD code and added missing NetBSD bits to ToolChain::getOSLibName(). The latter produced setup issues with the prebuilt LLVM toolchain, as the directory name with compiler-rt goodies were located in a path like ./lib/clang/7.0.0/lib/netbsd8.99.12 with a varying OS version. This could stop working after upgrades, so I've simplified it to "netbsd", similar to FreeBSD and Solaris.

  • Prebuilt toolchain for testers

I've prepared a build of Clang/LLVM with LLDB and compiler-rt features prebuilt on NetBSD/amd64 v. 8.99.12:

llvm-clang-compilerrt-lldb-7.0.0beta_2018-02-28.tar.bz2

  • Plan for the next milestone

With the approaching NetBSD 8.0 release I plan to finish backporting a few changes there from HEAD:

  • Remove one unused feature from ptrace(2), PTSETSIGMASK & PTGETSIGMASK. I've originally introduced these operations with criu/rr-like software in mind, but they are misusing or even abusing ptrace(2) and are not regular process debuggers. I plan to remove this operation from HEAD and backport this to NetBSD-8(BETA), before the release, so no compat will be required for this call. Future ports of criu/rr should involve dedicated kernel support for such requirements.
    Finish the backport of UCMACHINE_FP() to NetBSD-8. This will allow use of the same code in sanitizers in HEAD and NetBSD-8.0.
  • By popular demand, improve the regnsub(3) and regasub(3) API, adding support for more or less substitutions than 10.

Once done, I will return to ptrace(2) debugging and corrections.

DigitalOcean

Working with the NetBSD kernel

  • Overview

When working on complex systems, such as OS kernels, your attention span and cognitive energy are too valuable to be wasted on inefficiencies pertaining to ancillary tasks. After experimenting with different environmental setups for kernel debugging, some of which were awkward and distracting from my main objectives, I have arrived to my current workflow, which is described here. This approach is mainly oriented towards security research and the study of kernel internals.

Before delving into the details, this is the general outline of my environment:

My host system runs Linux. My target system is a QEMU guest.

I’m tracing and debugging on my host system by attaching GDB (with NetBSD x86-64 ABI support) to QEMU’s built-in GDB server.
I work with NetBSD-current. All sources are built on my host system with the cross-compilation toolchain produced by build.sh.
I use NFS to share the source tree and the build artifacts between the target and the host.
I find IDEs awkward, so for codebase navigation I mainly rely on vim, tmux and ctags.
For non-intrusive instrumentation, such as figuring out control flow, I’m using dtrace.

  • Preparing the host system

    • QEMU
    • GDB
    • NFS Exports
  • Building NetBSD-current

  • A word of warning

    • Now is a great time to familiarize yourself with the build.sh tool and its options. Be especially carefull with the following options:


-r Remove contents of TOOLDIR and DESTDIR before building.
-u Set MKUPDATE=yes; do not run "make clean" first.
Without this, everything is rebuilt, including the tools.

Chance are, you do not want to use these options once you’ve successfully built the cross-compilation toolchain and your entire userland, because building those takes time and there aren’t many good reasons to recompile them from scratch. Here’s what to expect:

On my desktop, running a quad-core Intel i5-3470 at 3.20GHz with 24GB of RAM and underlying directory structure residing on a SSD drive, the entire process took about 55 minutes. I was running make with -j12, so the machine was quite busy.
On an old Dell D630 laptop, running Intel Core 2 Duo T7500 at 2.20GHz with 4GB of RAM and a slow hard drive (5400RPM), the process took approximatelly 2.5 hours. I was running make with -j4. Based on the temperature alerts and CPU clock throttling messages, it was quite a struggle.

  • Acquiring the sources
  • Compiling the sources

    • Preparing the guest system
  • Provisioning your guest
  • Pkgin and NFS shares
  • Tailoring the kernel for debugging
  • Installing the new kernel
  • Configuring DTrace
  • Debugging the guest’s kernel

News Roundup

Add support for the experimental Internet-Draft "TCP Alternative Backoff”

```
Add support for the experimental Internet-Draft "TCP Alternative Backoff with
ECN (ABE)" proposal to the New Reno congestion control algorithm module.
ABE reduces the amount of congestion window reduction in response to
ECN-signalled congestion relative to the loss-inferred congestion response.

More details about ABE can be found in the Internet-Draft:
https://tools.ietf.org/html/draft-ietf-tcpm-alternativebackoff-ecn

The implementation introduces four new sysctls:

  • net.inet.tcp.cc.abe defaults to 0 (disabled) and can be set to non-zero to
    enable ABE for ECN-enabled TCP connections.

  • net.inet.tcp.cc.newreno.beta and net.inet.tcp.cc.newreno.betaecn set the
    multiplicative window decrease factor, specified as a percentage, applied to
    the congestion window in response to a loss-based or ECN-based congestion
    signal respectively. They default to the values specified in the draft i.e.
    beta=50 and beta
    ecn=80.

  • net.inet.tcp.cc.abe_frlossreduce defaults to 0 (disabled) and can be set to
    non-zero to enable the use of standard beta (50% by default) when repairing
    loss during an ECN-signalled congestion recovery episode. It enables a more
    conservative congestion response and is provided for the purposes of
    experimentation as a result of some discussion at IETF 100 in Singapore.

The values of beta and betaecn can also be set per-connection by way of the
TCP
CCALGOOPT TCP-level socket option and the new CCNEWRENOBETA or
CCNEWRENOBETA_ECN CC algo sub-options.

Submitted by: Tom Jones tj@enoti.me
Tested by: Tom Jones tj@enoti.me, Grenville Armitage garmitage@swin.edu.au
Relnotes: Yes
Differential Revision: https://reviews.freebsd.org/D11616
```

Meltdown-mitigation syspatch/errata now available

The recent changes in -current mitigating the Meltdown vulnerability have been backported to the 6.1 and 6.2 (amd64) releases, and the syspatch update (for 6.2) is now available.

```
Changes by: bluhm@cvs.openbsd.org 2018/02/26 05:36:18
Log message:
Implement a workaround against the Meltdown flaw in Intel CPUs.
The following changes have been backported from OpenBSD -current.

Changes by: guenther@cvs.openbsd.org 2018/01/06 15:03:13
Log message:
Handle %gs like %[def]s and reset set it in cpu_switchto() instead of on
every return to userspace.

Changes by: mlarkin@cvs.openbsd.org 2018/01/06 18:08:20
Log message:
Add identcpu.c and specialreg.h definitions for the new Intel/AMD MSRs
that should help mitigate spectre. This is just the detection piece, these
features are not yet used.
Part of a larger ongoing effort to mitigate meltdown/spectre. i386 will
come later; it needs some machdep.c cleanup first.

Changes by: mlarkin@cvs.openbsd.org 2018/01/07 12:56:19
Log message:
remove all PG_G global page mappings from the kernel when running on
Intel CPUs. Part of an ongoing set of commits to mitigate the Intel
"meltdown" CVE. This diff does not confer any immunity to that
vulnerability - subsequent commits are still needed and are being
worked on presently.
ok guenther, deraadt

Changes by: mlarkin@cvs.openbsd.org 2018/01/12 01:21:30
Log message:
IBRS -> IBRS,IBPB in identifycpu lines

Changes by: guenther@cvs.openbsd.org 2018/02/21 12:24:15
Log message:
Meltdown: implement user/kernel page table separation.
On Intel CPUs which speculate past user/supervisor page permission checks,
use a separate page table for userspace with only the minimum of kernel code
and data required for the transitions to/from the kernel (still marked as
supervisor-only, of course):
- the IDT (RO)
- three pages of kernel text in the .kutext section for interrupt, trap,
and syscall trampoline code (RX)
- one page of kernel data in the .kudata section for TLB flush IPIs (RW)
- the lapic page (RW, uncachable)
- per CPU: one page for the TSS+GDT (RO) and one page for trampoline
stacks (RW)
When a syscall, trap, or interrupt takes a CPU from userspace to kernel the
trampoline code switches page tables, switches stacks to the thread's real
kernel stack, then copies over the necessary bits from the trampoline stack.
On return to userspace the opposite occurs: recreate the iretq frame on the
trampoline stack, switch stack, switch page tables, and return to userspace.
mlarkin@ implemented the pmap bits and did 90% of the debugging, diagnosing
issues on MP in particular, and drove the final push to completion.
Many rounds of testing by naddy@, sthen@, and others
Thanks to Alex Wilson from Joyent for early discussions about trampolines
and their data requirements.
Per-CPU page layout mostly inspired by DragonFlyBSD.
ok mlarkin@ deraadt@

Changes by: bluhm@cvs.openbsd.org 2018/02/22 13:18:59
Log message:
The GNU assembler does not understand 1ULL, so replace the constant
with 1. Then it compiles with gcc, sign and size do not matter
here.

Changes by: bluhm@cvs.openbsd.org 2018/02/22 13:27:14
Log message:
The compile time assertion for cpu info did not work with gcc.
Rephrase the condition in a way that both gcc and clang accept it.

Changes by: guenther@cvs.openbsd.org 2018/02/22 13:36:40
Log message:
Set the PG_G (global) bit on the special page table entries that are shared
between the u-k and u+k tables, because they're actually in all tables.

OpenBSD 6.1 errata 037
```

  • 6.2

```
Changes by: bluhm@cvs.openbsd.org 2018/02/26 05:29:48
Log message:
Implement a workaround against the Meltdown flaw in Intel CPUs.
The following changes have been backported from OpenBSD -current.

Changes by: guenther@cvs.openbsd.org 2018/01/06 15:03:13
Log message:
Handle %gs like %[def]s and reset set it in cpu_switchto() instead of on
every return to userspace.

Changes by: mlarkin@cvs.openbsd.org 2018/01/06 18:08:20
Log message:
Add identcpu.c and specialreg.h definitions for the new Intel/AMD MSRs
that should help mitigate spectre. This is just the detection piece, these
features are not yet used.
Part of a larger ongoing effort to mitigate meltdown/spectre. i386 will
come later; it needs some machdep.c cleanup first.

Changes by: mlarkin@cvs.openbsd.org 2018/01/07 12:56:19
Log message:
remove all PG_G global page mappings from the kernel when running on
Intel CPUs. Part of an ongoing set of commits to mitigate the Intel
"meltdown" CVE. This diff does not confer any immunity to that
vulnerability - subsequent commits are still needed and are being
worked on presently.

Changes by: mlarkin@cvs.openbsd.org 2018/01/12 01:21:30
Log message:
IBRS -> IBRS,IBPB in identifycpu lines

Changes by: guenther@cvs.openbsd.org 2018/02/21 12:24:15
Log message:
Meltdown: implement user/kernel page table separation.
On Intel CPUs which speculate past user/supervisor page permission checks,
use a separate page table for userspace with only the minimum of kernel code
and data required for the transitions to/from the kernel (still marked as
supervisor-only, of course):
- the IDT (RO)
- three pages of kernel text in the .kutext section for interrupt, trap,
and syscall trampoline code (RX)
- one page of kernel data in the .kudata section for TLB flush IPIs (RW)
- the lapic page (RW, uncachable)
- per CPU: one page for the TSS+GDT (RO) and one page for trampoline
stacks (RW)
When a syscall, trap, or interrupt takes a CPU from userspace to kernel the
trampoline code switches page tables, switches stacks to the thread's real
kernel stack, then copies over the necessary bits from the trampoline stack.
On return to userspace the opposite occurs: recreate the iretq frame on the
trampoline stack, switch stack, switch page tables, and return to userspace.
mlarkin@ implemented the pmap bits and did 90% of the debugging, diagnosing
issues on MP in particular, and drove the final push to completion.
Many rounds of testing by naddy@, sthen@, and others
Thanks to Alex Wilson from Joyent for early discussions about trampolines
and their data requirements.
Per-CPU page layout mostly inspired by DragonFlyBSD.

Changes by: bluhm@cvs.openbsd.org 2018/02/22 13:18:59
Log message:
The GNU assembler does not understand 1ULL, so replace the constant
with 1. Then it compiles with gcc, sign and size do not matter
here.

Changes by: bluhm@cvs.openbsd.org 2018/02/22 13:27:14
Log message:
The compile time assertion for cpu info did not work with gcc.
Rephrase the condition in a way that both gcc and clang accept it.

Changes by: guenther@cvs.openbsd.org 2018/02/22 13:36:40
Log message:
Set the PG_G (global) bit on the special page table entries that are shared
between the u-k and u+k tables, because they're actually in all tables.

OpenBSD 6.2 errata 009
```

iXsystems

a2k18 Hackathon Report: Ken Westerback on dhclient and more

Ken Westerback (krw@) has sent in the first report from the (recently concluded) a2k18 hackathon:

```

Whew.

Once in Dunedin the hacking commenced. The background was a regular tick of new meltdown diffs to test in addition to whatever work one was actually engaged in. I was lucky (?) in that none of the problems with the various versions cropped up on my laptop.
```

```
I worked with rpe@ and tb@ to make the install script create the 'correct' FQDN when dhclient was involved. I worked with tb@ on some code cleanup in various bits of the base. dhclient(8) got some nice cleanup, further pruning/improving log messages in particular. In addition the oddball -q option was flipped into the more normal -v. I.e. be quiet by default and verbose on request.

More substantially the use of recorded leases was made less intrusive by avoiding continual reconfiguration of the interface with the same information. The 'request', 'require' and 'ignore' dhclient.conf(5) statement were changed so they are cumulative, making it easier to build longer lists of affected options.

I tweaked softraid(4) to remove a handrolled version of duid_format().

I sprinkled a couple of M_WAITOK into amd64 and i386 mpbios to document that there is really no need to check for NULL being returned from some malloc() calls.

I continued to help test the new filesystem quiescing logic that deraadt@ committed during the hackathon.

I only locked myself out of my room once!

Fueled by the excellent coffee from local institutions The Good Earth Cafe and The Good Oil Cafe, and the excellent hacking facilities and accommodations at the University of Otago it was another enjoyable and productive hackathon south of the equator. And I even saw penguins.

Thanks to Jim Cheetham and the support from the project and the OpenBSD Foundation that made it all possible
```

Poetic License

I found this when going through old documents. It looks like I wrote it and never posted it. Perhaps I didn’t consider it finished at the time. But looking at it now, I think it’s good enough to share. It’s a redrafting of the BSD licence, in poetic form. Maybe I had plans to do other licences one day; I can’t remember.

I’ve interleaved it with the original license text so you can see how true, or otherwise, I’ve been to it. Enjoy :-)

```
Copyright (c) ,
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
```

You may redistribute and use –
as source or binary, as you choose,
and with some changes or without –
this software; let there be no doubt.
But you must meet conditions three,
if in compliance you wish to be.


1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. Neither the name of the nor the names of its
contributors may be used to endorse or promote products derived
from this software without specific prior written permission.

The first is obvious, of course –
To keep this text within the source.
The second is for binaries
Place in the docs a copy, please.
A moral lesson from this ode –
Don’t strip the copyright on code.

The third applies when you promote:
You must not take, from us who wrote,
our names and make it seem as true
we like or love your version too.
(Unless, of course, you contact us
And get our written assensus.)


THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.

One final point to be laid out
(You must forgive my need to shout):
THERE IS NO WARRANTY FOR THIS
WHATEVER THING MAY GO AMISS.
EXPRESS, IMPLIED, IT’S ALL THE SAME –
RESPONSIBILITY DISCLAIMED.

WE ARE NOT LIABLE FOR LOSS
NO MATTER HOW INCURRED THE COST
THE TYPE OR STYLE OF DAMAGE DONE
WHATE’ER THE LEGAL THEORY SPUN.
THIS STILL REMAINS AS TRUE IF YOU
INFORM US WHAT YOU PLAN TO DO.

When all is told, we sum up thus –
Do what you like, just don’t sue us.

Beastie Bits

Tarsnap ad

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Mar 29 2018

1hr 32mins

Play

Rank #2: Episode 261: FreeBSDcon Flashback | BSD Now 261

Podcast cover
Read more

Insight into TrueOS and Trident, stop evildoers with pf-badhost, Flashback to FreeBSDcon ‘99, OpenBSD’s measures against TLBleed, play Morrowind on OpenBSD in 5 steps, DragonflyBSD developers shocked at Threadripper performance, and more.

##Headlines
###An Insight into the Future of TrueOS BSD and Project Trident

Last month, TrueOS announced that they would be spinning off their desktop offering. The team behind the new project, named Project Trident, have been working furiously towards their first release. They did take a few minutes to answer some of our question about Project Trident and TrueOS. I would like to thank JT and Ken for taking the time to compile these answers.

  • It’s FOSS: What is Project Trident?

Project Trident: Project Trident is the continuation of the TrueOS Desktop. Essentially, it is the continuation of the primary “TrueOS software” that people have been using for the past 2 years. The continuing evolution of the entire TrueOS project has reached a stage where it became necessary to reorganize the project. To understand this change, it is important to know the history of the TrueOS project.

Originally, Kris Moore created PC-BSD. This was a Desktop release of FreeBSD focused on providing a simple and user-friendly graphical experience for FreeBSD. PC-BSD grew and matured over many years. During the evolution of PC-BSD, many users began asking for a server focused version of the software. Kris agreed, and TrueOS was born as a scaled down server version of PC-BSD. In late 2016, more contributors and growth resulted in significant changes to the PC-BSD codebase. Because the new development was so markedly different from the original PC-BSD design, it was decided to rebrand the project.

TrueOS was chosen as the name for this new direction for PC-BSD as the project had grown beyond providing only a graphical front to FreeBSD and was beginning to make fundamental changes to the FreeBSD operating system. One of these changes was moving PC-BSD from being based on each FreeBSD Release to TrueOS being based on the active and less outdated FreeBSD Current. Other major changes are using OpenRC for service management and being more aggressive about addressing long-standing issues with the FreeBSD release process. TrueOS moved toward a rolling release cycle, twice a year, which tested and merged FreeBSD changes directly from the developer instead of waiting months or even years for the FreeBSD review process to finish. TrueOS also deprecated and removed obsolete technology much more regularly.

As the TrueOS Project grew, the developers found these changes were needed by other FreeBSD-based projects. These projects began expressing interest in using TrueOS rather than FreeBSD as the base for their project. This demonstrated that TrueOS needed to again evolve into a distribution framework for any BSD project to use. This allows port maintainers and source developers from any BSD project to pool their resources and use the same source repositories while allowing every distribution to still customize, build, and release their own self-contained project. The result is a natural split of the traditional TrueOS team. There were now naturally two teams in the TrueOS project: those working on the build infrastructure and FreeBSD enhancements – the “core” part of the project, and those working on end-user experience and utility – the “desktop” part of the project.

When the decision was made to formally split the projects, the obvious question that arose was what to call the “Desktop” project. As TrueOS was already positioned to be a BSD distribution platform, the developers agreed the desktop side should pick a new name. There were other considerations too, one notable being that we were concerned that if we continued to call the desktop project “TrueOS Desktop”, it would prevent people from considering TrueOS as the basis for their distribution because of misconceptions that TrueOS was a desktop-focused OS. It also helps to “level the playing field” for other desktop distributions like GhostBSD so that TrueOS is not viewed as having a single “blessed” desktop version.

  • It’s FOSS: What features will TrueOS add to the FreeBSD base?

Project Trident: TrueOS has already added a number of features to FreeBSD:
OpenRC replaces rc.d for service management
LibreSSL in base
Root NSS certificates out-of-box
Scriptable installations (pc-sysinstall)
The full list of changes can be seen on the TrueOS repository (https://github.com/trueos/trueos/blob/trueos-master/README.md). This list does change quite regularly as FreeBSD development itself changes.

  • It’s FOSS: I understand that TrueOS will have a new feature that will make creating a desktop spin of TrueOS very easy. Could you explain that new feature?

Project Trident: Historically, one of the biggest hurdles for creating a desktop version of FreeBSD is that the build options for packages are tuned for servers rather than desktops. This means a desktop distribution cannot use the pre-built packages from FreeBSD and must build, use, and maintain a custom package repository. Maintaining a fork of the FreeBSD ports tree is no trivial task. TrueOS has created a full distribution framework so now all it takes to create a custom build of FreeBSD is a single JSON manifest file. There is now a single “source of truth” for the source and ports repositories that is maintained by the TrueOS team and regularly tagged with “stable” build markers. All projects can use this framework, which makes updates trivial.

  • It’s FOSS: Do you think that the new focus of TrueOS will lead to the creation of more desktop-centered BSDs?

Project Trident: That is the hope. Historically, creating a desktop-centered BSD has required a lot of specialized knowledge. Not only do most people not have this knowledge, but many do not even know what they need to learn until they start troubleshooting. TrueOS is trying to drastically simplify this process to enable the wider Open Source community to experiment, contribute, and enjoy BSD-based projects.

  • It’s FOSS: What is going to happen to TrueOS Pico? Will Project Trident have ARM support?

Project Trident: Project Trident will be dependent on TrueOS for ARM support. The developers have talked about the possibility of supporting ARM64 and RISC-V architectures, but it is not possible at the current time. If more Open Source contributors want to help develop ARM and RISC-V support, the TrueOS project is definitely willing to help test and integrate that code.

  • It’s FOSS: What does this change (splitting Trus OS into Project Trident) mean for the Lumina desktop environment?

Project Trident: Long-term, almost nothing. Lumina is still the desktop environment for Project Trident and will continue to be developed and enhanced alongside Project Trident just as it was for TrueOS. Short-term, we will be delaying the release of Lumina 2.0 and will release an updated version of the 1.x branch (1.5.0) instead. This is simply due to all the extra overhead to get Project Trident up and running. When things settle down into a rhythm, the development of Lumina will pick up once again.

  • It’s FOSS: Are you planning on including any desktop environments besides Lumina?

Project Trident: While Lumina is included by default, all of the other popular desktop environments will be available in the package repo exactly as they had been before.

  • It’s FOSS: Any plans to include Steam to increase the userbase?

Project Trident: Steam is still unavailable natively on FreeBSD, so we do not have any plans to ship it out of the box currently. In the meantime, we highly recommend installing the Windows version of Steam through the PlayOnBSD utility.

  • It’s FOSS: What will happen to the AppCafe?

Project Trident: The AppCafe is the name of the graphical interface for the “pkg” utility integrated into the SysAdm client created by TrueOS. This hasn’t changed. SysAdm, the graphical client, and by extension AppCafe are still available for all TrueOS-based distributions to use.

  • It’s FOSS: Does Project Trident have any corporate sponsors lined up? If not, would you be open to it or would you prefer that it be community supported?

Project Trident: iXsystems is the first corporate sponsor of Project Trident and we are always open to other sponsorships as well. We would prefer smaller individual contributions from the community, but we understand that larger project needs or special-purpose goals are much more difficult to achieve without allowing larger corporate sponsorships as well. In either case, Project Trident is always looking out for the best interests of the community and will not allow intrusive or harmful code to enter the project even if a company or individual tries to make that code part of a sponsorship deal.

  • It’s FOSS: BSD always seems to be lagging in terms of support for newer devices. Will TrueOS be able to remedy that with a quicker release cycle?

Project Trident: Yes! That was a primary reason for TrueOS to start tracking the CURRENT branch of FreeBSD in 2016. This allows for the changes that FreeBSD developers are making, including new hardware support, to be available much sooner than if we followed the FreeBSD release cycle.

  • It’s FOSS: Do you have any idea when Project Trident will have its first release?

Project Trident: Right now we are targeting a late August release date. This is because Project Trident is “kicking the wheels” on the new TrueOS distribution system. We want to ensure everything is working smoothly before we release. Going forward, we plan on having regular package updates every week or two for the end-user packages and a new release of Trident with an updated OS version every 6 months. This will follow the TrueOS release schedule with a small time offset.

###pf-badhost: Stop the evil doers in their tracks!

pf-badhost is a simple, easy to use badhost blocker that uses the power of the pf firewall to block many of the internet’s biggest irritants. Annoyances such as ssh bruteforcers are largely eliminated. Shodan scans and bots looking for webservers to abuse are stopped dead in their tracks. When used to filter outbound traffic, pf-badhost blocks many seedy, spooky malware containing and/or compromised webhosts.
Filtering performance is exceptional, as the badhost list is stored in a pf table. To quote the OpenBSD FAQ page regarding tables: “the lookup time on a table holding 50,000 addresses is only slightly more than for one holding 50 addresses.”
pf-badhost is simple and powerful. The blocklists are pulled from quality, trusted sources. The ‘Firehol’, ‘Emerging Threats’ and ‘Binary Defense’ block lists are used as they are popular, regularly updated lists of the internet’s most egregious offenders. The pf-badhost.sh script can easily be expanded to use additional or alternate blocklists.
pf-badhost works best when used in conjunction with unbound-adblock for the ultimate badhost blocking.

  • Notes:
  • If you are trying to run pf-badhost on a LAN or are using NAT, you will want to add a rule to your pf.conf appearing BEFORE the pf-badhost rules allowing traffic to and from your local subnet so that you can still access your gateway and any DNS servers.
  • Conversely, adding a line to pf-badhost.sh that removes your subnet range from the table should also work. Just make sure you choose a subnet range / CIDR block that is actually in the list. 192.168.0.0/16, 172.16.0.0/12 and 10.0.0.0/8 are the most common home/office subnet ranges.

DigitalOcean
https://do.co/bsdnow

###FLASHBACK: FreeBSDCon’99: Fans of Linux’s lesser-known sibling gather for the first time

FreeBSD, a port of BSD Unix to Intel, has been around almost as long as Linux has – but without the media hype. Its developer and user community recently got a chance to get together for the first time, and they did it in the city where BSD – the Berkeley Software Distribution – was born some 25 years ago.
October 17, 1999 marked a milestone in the history of FreeBSD – the first FreeBSD conference was held in the city where it all began, Berkeley, CA. Over 300 developers, users, and interested parties attended from around the globe.
This was easily 50 percent more people than the conference organizers had expected. This first conference was meant to be a gathering mostly for developers and FreeBSD advocates. The turnout was surprisingly (and gratifyingly) large.
In fact, attendance exceeded expectations so much that, for instance, Kirk McKusick had to add a second, identical tutorial on FreeBSD internals, because it was impossible for everyone to attend the first!
But for a first-ever conference, I was impressed by how smoothly everything seemed to go. Sessions started on time, and the sessions I attended were well-run; nothing seemed to be too cold, dark, loud, late, or off-center.
Of course, the best part about a conference such as this one is the opportunity to meet with other people who share similar interests. Lunches and breaks were a good time to meet people, as was the Tuesday night beer bash.
The Wednesday night reception was of a type unusual for the technical conferences I usually attend – a three-hour Hornblower dinner cruise on San Francisco Bay. Not only did we all enjoy excellent food and company, but we all got to go up on deck and watch the lights of San Francisco and Berkeley as we drifted by. Although it’s nice when a conference attracts thousands of attendees, there are some things that can only be done with smaller groups of people; this was one of them.
In short, this was a tiny conference, but a well-run one.

  • Sessions

Although it was a relatively small conference, the number and quality of the sessions belied the size. Each of the three days of the conference featured a different keynote speaker. In addition to Jordan Hubbard, Jeremy Allison spoke on “Samba Futures” on day two, and Brian Behlendorf gave a talk on “FreeBSD and Apache: A Perfect Combo” to start off the third day.
The conference sessions themselves were divided into six tracks: advocacy, business, development, networking, security, and panels. The panels track featured three different panels, made up of three different slices of the community: the FreeBSD core team, a press panel, and a prominent user panel with representatives from such prominent commercial users as Yahoo! and USWest.
I was especially interested in Apple Computer’s talk in the development track. Wilfredo Sanchez, technical lead for open source projects at Apple (no, that’s not an oxymoron!) spoke about Apple’s Darwin project, the company’s operating system road map, and the role of BSD (and, specifically, FreeBSD) in Apple’s plans.
Apple and Unix have had a long and uneasy history, from the Lisa through the A/UX project to today. Personally, I’m very optimistic about the chances for the Darwin project to succeed. Apple’s core OS kernel team has chosen FreeBSD as its reference platform. I’m looking forward to what this partnership will bring to both sides.
Other development track sessions included in-depth tutorials on writing device drivers, basics of the Vinum Volume Manager, Fibre Channel, development models (the open repository model), and the FreeBSD Documentation Project (FDP). If you’re interested in contributing to the FreeBSD project, the FDP is a good place to start.
Advocacy sessions included “How One Person Can Make a Difference” (a timeless topic that would find a home at any technical conference!) and “Starting and Managing A User Group” (trials and tribulations as well as rewards).
The business track featured speakers from three commercial users of FreeBSD: Cybernet, USWest, and Applix. Applix presented its port of Applixware Office for FreeBSD and explained how Applix has taken the core services of Applixware into open source.
Commercial applications and open source were once a rare combination; we can only hope the trend away from that state of affairs will continue.

  • Commercial use of FreeBSD

The use of FreeBSD in embedded applications is increasing as well – and it is increasing at the same rate that hardware power is. These days, even inexpensive systems are able to run a BSD kernel.
The BSD license and the solid TCP/IP stack prove significant enticements to this market as well. (Unlike the GNU Public License, the BSD license does not require that vendors make derivative works open source.)
Companies such as USWest and Verio use FreeBSD for a wide variety of different Internet services.
Yahoo! and Hotmail are examples of companies that use FreeBSD extensively for more specific purposes. Yahoo!, for example, has many hundreds of FreeBSD boxes, and Hotmail has almost 2000 FreeBSD machines at its data center in the San Francisco Bay area.
Hotmail is owned by Microsoft, so the fact that it runs FreeBSD is a secret. Don’t tell anyone…
When asked to comment on the increasing commercial interest in BSD, Hubbard said that FreeBSD is learning the Red Hat lesson. “Walnut Creek and others with business interests in FreeBSD have learned a few things from the Red Hat IPO,” he said, “and nobody is just sitting around now, content with business as usual. It’s clearly business as unusual in the open source world today.”
Hubbard had also singled out some of BSD’s commercial partners, such as Whistle Communications, for praise in his opening day keynote. These partners play a key role in moving the project forward, he said, by contributing various enhancements and major new systems, such as Netgraph, as well as by contributing paid employee time spent on FreeBSD.
Even short FreeBSD-related contacts can yield good results, Hubbard said. An example of this is the new jail() security code introduced in FreeBSD 3.x and 4.0, which was contributed by R & D Associates. A number of ISPs are also now donating the hardware and bandwidth that allows the project to provide more resource mirrors and experimental development sites.

  • See you next year

And speaking of corporate sponsors, thanks go to Walnut Creek for sponsoring the conference, and to Yahoo! for covering all the expenses involved in bringing the entire FreeBSD core team to Berkeley.
As a fan of FreeBSD, I’m happy to see that the project has finally produced a conference. It was time: many of the 16 core team members had been working together on a regular basis for nearly seven years without actually meeting face to face.
It’s been an interesting year for open source projects. I’m looking forward to the next year – and the next BSD conference – to be even better.

##News Roundup
###OpenBSD Recommends: Disable SMT/Hyperthreading in all Intel BIOSes

Two recently disclosed hardware bugs affected Intel cpus: - TLBleed - T1TF (the name "Foreshadow" refers to 1 of 3 aspects of this bug, more aspects are surely on the way) Solving these bugs requires new cpu microcode, a coding workaround, *AND* the disabling of SMT / Hyperthreading. SMT is fundamentally broken because it shares resources between the two cpu instances and those shared resources lack security differentiators. Some of these side channel attacks aren't trivial, but we can expect most of them to eventually work and leak kernel or cross-VM memory in common usage circumstances, even such as javascript directly in a browser. There will be more hardware bugs and artifacts disclosed. Due to the way SMT interacts with speculative execution on Intel cpus, I expect SMT to exacerbate most of the future problems. A few months back, I urged people to disable hyperthreading on all Intel cpus. I need to repeat that: DISABLE HYPERTHREADING ON ALL YOUR INTEL MACHINES IN THE BIOS. Also, update your BIOS firmware, if you can. OpenBSD -current (and therefore 6.4) will not use hyperthreading if it is enabled, and will update the cpu microcode if possible. But what about 6.2 and 6.3? The situation is very complex, continually evolving, and is taking too much manpower away from other tasks. Furthermore, Intel isn't telling us what is coming next, and are doing a terrible job by not publically documenting what operating systems must do to resolve the problems. We are having to do research by reading other operating systems. There is no time left to backport the changes -- we will not be issuing a complete set of errata and syspatches against 6.2 and 6.3 because it is turning into a distraction. Rather than working on every required patch for 6.2/6.3, we will re-focus manpower and make sure 6.4 contains the best solutions possible. So please try take responsibility for your own machines: Disable SMT in the BIOS menu, and upgrade your BIOS if you can. I'm going to spend my money at a more trustworthy vendor in the future.

###Get Morrowind running on OpenBSD in 5 simple steps

This article contains brief instructions on how to get one of the greatest Western RPGs of all time, The Elder Scrolls III: Morrowind, running on OpenBSD using the OpenMW open source engine recreation. These instructions were tested on a ThinkPad X1 Carbon Gen 3. The information was adapted from this OpenMW forum thread: https://forum.openmw.org/viewtopic.php?t=3510

    1. Purchase and download the DRM-free version from GOG (also considered the best version due to the high quality PDF guide that it comes with): https://www.gog.com/game/the_elder_scrolls_iii_morrowind_goty_edition
    1. Install the required packages built from the ports tree as root. openmw is the recreated game engine, and innoextract is how we will get the game data files out of the win32 executable.

pkg_add openmw innoextract

    1. Move the file from GOG setup_tes_morrowind_goty_2.0.0.7.exe into its own directory morrowind/ due to innoextract’s default behaviour of extracting into the current directory. Then type:

innoextract setup_tes_morrowind_goty_2.0.0.7.exe

    1. Type openmw-wizard and follow the straightforward instructions. Note that you have a pre-existing installation, and select the morrowind/app/Data Files folder that innoextract extracted.
    1. Type in openmw-launcher, toggle the settings to your preferences, and then hit play!

iXsystems
https://twitter.com/allanjude/status/1034647571124367360

###My First Clang Bug

Part of the role of being a packager is compiling lots (and lots) of packages. That means compiling lots of code from interesting places and in a variety of styles. In my opinion, being a good packager also means providing feedback to upstream when things are bad. That means filing upstream bugs when possible, and upstreaming patches.
One of the “exciting” moments in packaging is when tools change. So each and every major CMake update is an exercise in recompiling 2400 or more packages and adjusting bits and pieces. When a software project was last released in 2013, adjusting it to modern tools can become quite a chore (e.g. Squid Report Generator). CMake is excellent for maintaining backwards compatibility, generally accommodating old software with new policies. The most recent 3.12 release candidate had three issues filed from the FreeBSD side, all from fallout with older software. I consider the hours put into good bug reports, part of being a good citizen of the Free Software world.
My most interesting bug this week, though, came from one line of code somewhere in Kleopatra: Q_UNUSED(gpgagent_data);
That one line triggered a really peculiar link error in KDE’s FreeBSD CI system. Yup … telling the compiler something is unused made it fall over. Commenting out that line got rid of the link error, but introduced a warning about an unused function. Working with KDE-PIM’s Volker Krause, we whittled the problem down to a six-line example program — two lines if you don’t care much for coding style. I’m glad, at that point, that I could throw it over the hedge to the LLVM team with some explanatory text. Watching the process on their side reminds me ever-so-strongly of how things work in KDE (or FreeBSD for that matter): Bugzilla, Phabricator, and git combine to be an effective workflow for developers (perhaps less so for end-users).
Today I got a note saying that the issue had been resolved. So brief a time for a bug. Live fast. Get squashed young.

###DragonFlyBSD Now Runs On The Threadripper 2990WX, Developer Shocked At Performance

Last week I carried out some tests of BSD vs. Linux on the new 32-core / 64-thread Threadripper 2990WX. I tested FreeBSD 11, FreeBSD 12, and TrueOS – those benchmarks will be published in the next few days. I tried DragonFlyBSD, but at the time it wouldn’t boot with this AMD HEDT processor. But now the latest DragonFlyBSD development kernel can handle the 2990WX and the lead DragonFly developer calls this new processor “a real beast” and is stunned by its performance potential.

When I tried last week, the DragonFlyBSD 5.2.2 stable release nor DragonFlyBSD 5.3 daily snapshot would boot on the 2990WX. But it turns out Matthew Dillon, the lead developer of DragonFlyBSD, picked up a rig and has it running now. So in time for the next 5.4 stable release or those using the daily snapshots can have this 32-core / 64-thread Zen+ CPU running on this operating system long ago forked from FreeBSD.

In announcing his success in bringing up the 2990WX under DragonFlyBSD, which required a few minor changes, he shared his performance thoughts and hopes for the rig. “The cpu is a real beast, packing 32 cores and 64 threads. It blows away our dual-core Xeon to the tune of being +50% faster in concurrent compile tests, and it also blows away our older 4-socket Opteron (which we call ‘Monster’) by about the same margin. It’s an impressive CPU. For now the new beast is going to be used to help us improve I/O performance through the filesystem, further SMP work (but DFly scales pretty well to 64 threads already), and perhaps some driver to work to support the 10gbe on the mobo.”

Dillon shared some results on the system as well. " The Threadripper 2990WX is a beast. It is at least 50% faster than both the quad socket opteron and the dual socket Xeon system I tested against. The primary limitation for the 2990WX is likely its 4 channels of DDR4 memory, and like all Zen and Zen+ CPUs, memory performance matters more than CPU frequency (and costs almost no power to pump up the performance). That said, it still blow away a dual-socket Xeon with 3x the number of memory channels. That is impressive!"

The well known BSD developer also added, “This puts the 2990WX at par efficiency vs a dual-socket Xeon system, and better than the dual-socket Xeon with slower memory and a power cap. This is VERY impressive. I should note that the 2990WX is more specialized with its asymetric NUMA architecture and 32 cores. I think the sweet spot in terms of CPU pricing and efficiency is likely going to be with the 2950X (16-cores/32-threads). It is clear that the 2990WX (32-cores/64-threads) will max out 4-channel memory bandwidth for many workloads, making it a more specialized part. But still awesome…This thing is an incredible beast, I’m glad I got it.”

While I have the FreeBSD vs. Linux benchmarks from a few days ago, it looks like now on my ever growing TODO list will be re-trying out the newest DragonFlyBSD daily snapshot for seeing how the performance compares in the mix. Stay tuned for the numbers that should be in the next day or two.

##Beastie Bits

Tarsnap

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Aug 30 2018

1hr 49mins

Play

Rank #3: Episode 277: Nmap Level Up | BSD Now 277

Podcast cover
Read more

The Open Source midlife crisis, Donald Knuth The Yoda of Silicon Valley, Certbot For OpenBSD's httpd, how to upgrade FreeBSD from 11 to 12, level up your nmap game, NetBSD desktop, and more.

##Headlines
###Open Source Confronts its midlife crisis

Midlife is tough: the idealism of youth has faded, as has inevitably some of its fitness and vigor. At the same time, the responsibilities of adulthood have grown. Making things more challenging, while you are navigating the turbulence of teenagers, your own parents are likely entering life’s twilight, needing help in new ways from their adult children. By midlife, in addition to the singular joys of life, you have also likely experienced its terrible sorrows: death, heartbreak, betrayal. Taken together, the fading of youth, the growth in responsibility and the endurance of misfortune can lead to cynicism or (worse) drastic and poorly thought-out choices. Add in a little fear of mortality and some existential dread, and you have the stuff of which midlife crises are made…
I raise this not because of my own adventures at midlife, but because it is clear to me that open source — now several decades old and fully adult — is going through its own midlife crisis. This has long been in the making: for years, I (and others) have been critical of service providers’ parasitic relationship with open source, as cloud service providers turn open source software into a service offering without giving back to the communities upon which they implicitly depend. At the same time, open source has been (rightfully) entirely unsympathetic to the proprietary software models that have been burned to the ground — but also seemingly oblivious as to the larger economic waves that have buoyed them.
So it seemed like only a matter of time before the companies built around open source software would have to confront their own crisis of confidence: open source business models are really tough, selling software-as-a-service is one of the most natural of them, the cloud service providers are really good at it — and their commercial appetites seem boundless. And, like a new cherry red two-seater sports car next to a minivan in a suburban driveway, some open source companies are dealing with this crisis exceptionally poorly: they are trying to restrict the way that their open source software can be used. These companies want it both ways: they want the advantages of open source — the community, the positivity, the energy, the adoption, the downloads — but they also want to enjoy the fruits of proprietary software companies in software lock-in and its monopolistic rents. If this were entirely transparent (that is, if some bits were merely being made explicitly proprietary), it would be fine: we could accept these companies as essentially proprietary software companies, albeit with an open source loss-leader. But instead, these companies are trying to license their way into this self-contradictory world: continuing to claim to be entirely open source, but perverting the license under which portions of that source are available. Most gallingly, they are doing this by hijacking open source nomenclature. Of these, the laughably named commons clause is the worst offender (it is plainly designed to be confused with the purely virtuous creative commons), but others (including CockroachDB’s Community License, MongoDB’s Server Side Public License, and Confluent’s Community License) are little better. And in particular, as it apparently needs to be said: no, “community” is not the opposite of “open source” — please stop sullying its good name by attaching it to licenses that are deliberately not open source! But even if they were more aptly named (e.g. “the restricted clause” or “the controlled use license” or — perhaps most honest of all — “the please-don’t-put-me-out-of-business-during-the-next-reInvent-keynote clause”), these licenses suffer from a serious problem: they are almost certainly asserting rights that the copyright holder doesn’t in fact have.
If I sell you a book that I wrote, I can restrict your right to read it aloud for an audience, or sell a translation, or write a sequel; these restrictions are rights afforded the copyright holder. I cannot, however, tell you that you can’t put the book on the same bookshelf as that of my rival, or that you can’t read the book while flying a particular airline I dislike, or that you aren’t allowed to read the book and also work for a company that competes with mine. (Lest you think that last example absurd, that’s almost verbatim the language in the new Confluent Community (sic) License.) I personally think that none of these licenses would withstand a court challenge, but I also don’t think it will come to that: because the vendors behind these licenses will surely fear that they wouldn’t survive litigation, they will deliberately avoid inviting such challenges. In some ways, this netherworld is even worse, as the license becomes a vessel for unverifiable fear of arbitrary liability.
let me put this to you as directly as possible: cloud services providers are emphatically not going to license your proprietary software. I mean, you knew that, right? The whole premise with your proprietary license is that you are finding that there is no way to compete with the operational dominance of the cloud services providers; did you really believe that those same dominant cloud services providers can’t simply reimplement your LDAP integration or whatever? The cloud services providers are currently reproprietarizing all of computing — they are making their own CPUs for crying out loud! — reimplementing the bits of your software that they need in the name of the service that their customers want (and will pay for!) won’t even move the needle in terms of their effort.
Worse than all of this (and the reason why this madness needs to stop): licenses that are vague with respect to permitted use are corporate toxin. Any company that has been through an acquisition can speak of the peril of the due diligence license audit: the acquiring entity is almost always deep pocketed and (not unrelatedly) risk averse; the last thing that any company wants is for a deal to go sideways because of concern over unbounded liability to some third-party knuckle-head. So companies that engage in license tomfoolery are doing worse than merely not solving their own problem: they are potentially poisoning the wellspring of their own community.
in the end, open source will survive its midlife questioning just as people in midlife get through theirs: by returning to its core values and by finding rejuvenation in its communities. Indeed, we can all find solace in the fact that while life is finite, our values and our communities survive us — and that our engagement with them is our most important legacy.

  • See the article for the rest

###Donald Knuth - The Yoda of Silicon Valley

For half a century, the Stanford computer scientist Donald Knuth, who bears a slight resemblance to Yoda — albeit standing 6-foot-4 and wearing glasses — has reigned as the spirit-guide of the algorithmic realm.
He is the author of “The Art of Computer Programming,” a continuing four-volume opus that is his life’s work. The first volume debuted in 1968, and the collected volumes (sold as a boxed set for about $250) were included by American Scientist in 2013 on its list of books that shaped the last century of science — alongside a special edition of “The Autobiography of Charles Darwin,” Tom Wolfe’s “The Right Stuff,” Rachel Carson’s “Silent Spring” and monographs by Albert Einstein, John von Neumann and Richard Feynman.
With more than one million copies in print, “The Art of Computer Programming” is the Bible of its field. “Like an actual bible, it is long and comprehensive; no other book is as comprehensive,” said Peter Norvig, a director of research at Google. After 652 pages, volume one closes with a blurb on the back cover from Bill Gates: “You should definitely send me a résumé if you can read the whole thing.”
The volume opens with an excerpt from “McCall’s Cookbook”:

Here is your book, the one your thousands of letters have asked us to publish. It has taken us years to do, checking and rechecking countless recipes to bring you only the best, only the interesting, only the perfect.

Inside are algorithms, the recipes that feed the digital age — although, as Dr. Knuth likes to point out, algorithms can also be found on Babylonian tablets from 3,800 years ago. He is an esteemed algorithmist; his name is attached to some of the field’s most important specimens, such as the Knuth-Morris-Pratt string-searching algorithm. Devised in 1970, it finds all occurrences of a given word or pattern of letters in a text — for instance, when you hit Command+F to search for a keyword in a document.
Now 80, Dr. Knuth usually dresses like the youthful geek he was when he embarked on this odyssey: long-sleeved T-shirt under a short-sleeved T-shirt, with jeans, at least at this time of year. In those early days, he worked close to the machine, writing “in the raw,” tinkering with the zeros and ones.

  • See the article for the rest

##News Roundup
###Let’s Encrypt: Certbot For OpenBSD’s httpd

  • Intro

Let’s Encrypt is “a free, automated, and open Certificate Authority”.
Certbot is “an easy-to-use automatic client that fetches and deploys SSL/TLS certificates for your web server”, well known as “the official Let’s Encrypt client”.
I remember well how excited I felt when I read Let’s Encrypt’s “Our First Certificate Is Now Live” in 2015.
How wonderful the goal of them is; it’s to “give people the digital certificates they need in order to enable HTTPS (SSL/TLS) for websites, for free” “to create a more secure and privacy-respecting Web”!
Since this year, they have begun to support even ACME v2 and Wildcard Certificate!
Well, in OpenBSD as well as other operating systems, it’s easy and comfortable to have their big help 😊

  • Environment
  • OS: OpenBSD 6.4 amd64
  • Web Server: OpenBSD’s httpd
  • Certification: Let’s Encrypt with Certbot 0.27
  • Reference: OpenBSD’s httpd

###FreeBSD 12 released: Here is how to upgrade FreeBSD 11 to 12

The FreeBSD project announces the availability of FreeBSD 12.0-RELEASE. It is the first release of the stable/12 branch. The new version comes with updated software and features for a wild variety of architectures. The latest release provides performance improvements and better support for FreeBSD jails and more. One can benefit greatly using an upgraded version of FreeBSD.

FreeBSD 12.0 supports amd64, i386, powerpc, powerpc64, powerpcspe, sparc64, armv6, armv7, and aarch64 architectures. One can run it on a standalone server or desktop system. Another option is to run it on Raspberry PI computer. FreeBSD 12 also runs on popular cloud service providers such as AWS EC2/Lightsail or Google compute VM.

  • New features and highlights:

  • OpenSSL version 1.1.1a (LTS)

  • OpenSSH server 7.8p1

  • Unbound server 1.8.1

  • Clang and co 6.0.1

  • The FreeBSD installer supports EFI+GELI as an installation option

  • VIMAGE FreeBSD kernel configuration option has been enabled by default. VIMAGE was the main reason I custom compiled FreeBSD for the last few years. No more custom compile for me.

  • Graphics drivers for modern ATI/AMD and Intel graphics cards are now available in the FreeBSD ports collection

  • ZFS has been updated to include new sysctl(s), vfs.zfs.arc_min_prefetch_ms and vfs.zfs.arc_min_prescient_prefetch_ms, which improve performance of the zpool scrub subcommand

  • The pf packet filter is now usable within a jail using vnet

  • KDE updated to version 5.12.5

  • The NFS version 4.1 includes pNFS server support

  • Perl 5.26.2

  • The default PAGER now defaults to less for most commands

  • The dd utility has been updated to add the status=progress option to match GNU/Linux dd command to show progress bar while running dd

  • FreeBSD now supports ext4 for read/write operation

  • Python 2.7

  • much more

###Six Ways to Level Up Your nmap Game

nmap is a network exploration tool and security / port scanner.
If you’ve heard of it, and you’re like me, you’ve most likely used it like this:
ie, you’ve pointed it at an IP address and observed the output which tells you the open ports on a host.
I used nmap like this for years, but only recently grokked the manual to see what else it could do. Here’s a quick look and some of the more useful things I found out.

    1. Scan a Network
    1. Scan All Ports
    1. Get service versions
    1. Use -A for more data
    1. Find out what nmap is up to
    1. Script your own scans with NSE

###[NetBSD Desktop]

##Beastie Bits

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Dec 24 2018

1hr 16mins

Play

Rank #4: Episode 262: OpenBSD Surfacing | BSD Now 262

Podcast cover
Read more

OpenBSD on Microsoft Surface Go, FreeBSD Foundation August Update, What’s taking so long with Project Trident, pkgsrc config file versioning, and MacOS remnants in ZFS code.

##Headlines
###OpenBSD on the Microsoft Surface Go

For some reason I like small laptops and the constraints they place on me (as long as they’re still usable). I used a Dell Mini 9 for a long time back in the netbook days and was recently using an 11" MacBook Air as my primary development machine for many years. Recently Microsoft announced a smaller, cheaper version of its Surface tablets called Surface Go which piqued my interest.

  • Hardware

The Surface Go is available in two hardware configurations: one with 4Gb of RAM and a 64Gb eMMC, and another with 8Gb of RAM with a 128Gb NVMe SSD. (I went with the latter.) Both ship with an Intel Pentium Gold 4415Y processor which is not very fast, but it’s certainly usable.
The tablet measures 9.65" across, 6.9" tall, and 0.3" thick. Its 10" diagonal 3:2 touchscreen is covered with Gorilla Glass and has a resolution of 1800x1200. The bezel is quite large, especially for such a small screen, but it makes sense on a device that is meant to be held, to avoid accidental screen touches.
The keyboard and touchpad are located on a separate, removable slab called the Surface Go Signature Type Cover which is sold separately. I opted for the “cobalt blue” cover which has a soft, cloth-like alcantara material. The cover attaches magnetically along the bottom edge of the device and presents USB-attached keyboard and touchpad devices. When the cover is folded up against the screen, it sends an ACPI sleep signal and is held to the screen magnetically. During normal use, the cover can be positioned flat on a surface or slightly raised up about 3/4" near the screen for better ergonomics. When using the device as a tablet, the cover can be rotated behind the screen which causes it to automatically stop sending keyboard and touchpad events until it is rotated back around.
The keyboard has a decent amount of key travel and a good layout, with Home/End/Page Up/Page Down being accessible via Fn+Left/Right/Up/Down but also dedicated Home/End/Page Up/Page Down keys on the F9-F12 keys which I find quite useful since the keyboard layout is somewhat small. By default, the F1-F12 keys do not send F1-F12 key codes and Fn must be used, either held down temporarily or Fn pressed by itself to enable Fn-lock which annoyingly keeps the bright Fn LED illuminated. The keys are backlit with three levels of adjustment, handled by the keyboard itself with the F7 key.
The touchpad on the Type Cover is a Windows Precision Touchpad connected via USB HID. It has a decent click feel but when the cover is angled up instead of flat on a surface, it sounds a bit hollow and cheap.

  • Surface Go Pen

The touchscreen is powered by an Elantech chip connected via HID-over-i2c, which also supports pen input. A Surface Pen digitizer is available separately from Microsoft and comes in the same colors as the Type Covers. The pen works without any pairing necessary, though the top button on it works over Bluetooth so it requires pairing to use. Either way, the pen requires an AAAA battery inside it to operate. The Surface Pen can attach magnetically to the left side of the screen when not in use.
A kickstand can swing out behind the display to use the tablet in a laptop form factor, which can adjust to any angle up to about 170 degrees. The kickstand stays firmly in place wherever it is positioned, which also means it requires a bit of force to pull it out when initially placing the Surface Go on a desk.
Along the top of the display are a power button and physical volume rocker buttons. Along the right side are the 3.5mm headphone jack, USB-C port, power port, and microSD card slot located behind the kickstand.
Charging can be done via USB-C or the dedicated charge port, which accommodates a magnetically-attached, thin barrel similar to Apple’s first generation MagSafe adapter. The charging cable has a white LED that glows when connected, which is kind of annoying since it’s near the mid-line of the screen rather than down by the keyboard. Unlike Apple’s MagSafe, the indicator light does not indicate whether the battery is charged or not. The barrel charger plug can be placed up or down, but in either direction I find it puts an awkward strain on the power cable coming out of it due to the vertical position of the port.
Wireless connectivity is provided by a Qualcomm Atheros QCA6174 802.11ac chip which also provides Bluetooth connectivity.
Most of the sensors on the device such as the gyroscope and ambient light sensor are connected behind an Intel Sensor Hub PCI device, which provides some power savings as the host CPU doesn’t have to poll the sensors all the time.

  • Firmware

The Surface Go’s BIOS/firmware menu can be entered by holding down the Volume Up button, then pressing and releasing the Power button, and releasing Volume Up when the menu appears. Secure Boot as well as various hardware components can be disabled in this menu. Boot order can also be adjusted. A temporary boot menu can be brought up the same way but using Volume Down instead.

###FreeBSD Foundation Update, August 2018

  • MESSAGE FROM THE EXECUTIVE DIRECTOR

Dear FreeBSD Community Member,
It’s been a busy summer for the Foundation. From traveling around the globe spreading the word about FreeBSD to bringing on new team members to improve the Project’s Continuous Integration work, we’re very excited about what we’ve accomplished. Take a minute to check out the latest updates within our Foundation sponsored projects; read more about our advocacy efforts in Bangladesh and community building in Cambridge; don’t miss upcoming Travel Grant deadlines, and new Developer Summits; and be sure to find out how your support will ensure our progress continues into 2019.
We can’t do this without you! Happy reading!! Deb

  • August 2018 Development Projects Update
  • Fundraising Update: Supporting the Project
  • August 2018 Release Engineering Update
  • BSDCam 2018 Recap
  • October 2018 FreeBSD Developer Summit Call for Participation
  • SANOG32 and COSCUP 2018 Recap
  • MeetBSD 2018 Travel Grant Application Deadline: September 7

##News Roundup
###Project Trident: What’s taking so long?

  • What is taking so long?

The short answer is that it’s complicated.
Project Trident is quite literally a test of the new TrueOS build system. As expected, there have been quite a few bugs, undocumented features, and other optional bits that we discovered we needed that were not initially present. All of these things have to be addressed and retested in a constant back and forth process.
While Ken and JT are both experienced developers, neither has done this kind of release engineering before. JT has done some release engineering back in his Linux days, but the TrueOS and FreeBSD build system is very different. Both Ken and JT are learning a completely new way of building a FreeBSD/TrueOS distribution. Please keep in mind that no one has used this new TrueOS build system before, so Ken and JT want to not only provide a good Trident release, but also provide a model or template for other potential TrueOS distributions too!

  • Where are we now?

Through perseverance, trial and error, and a lot of head-scratching we have reached the point of having successful builds. It took a while to get there, but now we are simply working out a few bugs with the new installer that Ken wrote as well as finding and fixing all the new Xorg configuration options which recently landed in FreeBSD. We also found that a number of services have been removed or replaced between TrueOS 18.03 and 18.06 so we are needing to adjust what we consider the “base” services for the desktop. All of these issues are being resolved and we are continually rebuilding and pulling in new patches from TrueOS as soon as they are committed.
In the meantime we have made an early BETA release of Trident available to the users in our Telegram Channel for those who want to help out in testing these early versions.

  • Do you foresee any other delays?

At the moment we are doing many iterations of testing and tweaking the install ISO and package configurations in order to ensure that all the critical functionality works out-of-box (networking, sound, video, basic apps, etc). While we do not foresee any other major delays, sometimes things happen that our outside of our control. For an example, one of the recent delays that hit recently was completely unexpected: we had a hard drive failure on our build server. Up until recently, The aptly named “Poseidon” build server was running a Micron m500dc drive, but that drive is now constantly reporting errors. Despite ordering a replacement Western Digital Blue SSD several weeks ago, we just received it this past week. The drive is now installed with the builder back to full functionality, but we did lose many precious days with the delay.
The build server for Project Trident is very similar to the one that JT donated to the TrueOS project. JT had another DL580 G7, so he donated one to the Trident Project for their build server. Poseidon also has 256GB RAM (64 x 4GB sticks) which is a smidge higher than what the TrueOS builder has.
Since we are talking about hardware, we probably should address another question we get often, “What Hardware are the devs testing on?” So let’s go ahead and answer that one now.

  • Developer Hardware

  • JT: His main test box is a custom-built Intel i7 7700K system running 32GB RAM, dual Intel Optane 900P drives, and an Nvidia 1070 GTX with four 4K Acer Monitors. He also uses a Lenovo x250 ThinkPad alongside a desk full of x230t and x220 ThinkPads. One of which he gave away at SouthEast LinuxFest this year, which you can read about here. However it’s not done there, being a complete hardware hoarder, JT also tests on several Intel NUCs and his second laptop a Fujitsu t904, not to mention a Plethora of HP DL580 servers, a DL980 server, and a stack of BL485c, BL460c, and BL490c Blades in his HP c7000 and c3000 Bladecenter chassis. (Maybe it’s time for an intervention for his hardware collecting habits)

  • Ken: For a laptop, he primarily uses a 3rd generation X1 Carbon, but also has an old Eee PC T101MT Netbook (dual core 1GHz, 2GB of memory) which he uses for verifying how well Trident works on low-end hardware. As far as workstations go, his office computer is an Intel i7 with an NVIDIA Geforce GTX 960 running three 4K monitors and he has a couple other custom-built workstations (1 AMD, 1 Intel+NVIDIA) at his home. Generally he assembled random workstations based on hardware that was given to him or that he could acquire cheap.

  • Tim: is using a third gen X1 Carbon and a custom built desktop with an Intel Core i5-4440 CPU, 16 GiB RAM, Nvidia GeForce GTX 750 Ti, and a RealTek 8168 / 8111 network card.

  • Rod: Rod uses… No one knows what Rod uses, It’s kinda like how many licks does it take to get to the center of a Tootsie-Roll Tootsie-Pop… the world may just never know.

###NetBSD GSoC: pkgsrc config file versioning

Packages may install code (both machine executable code and interpreted programs), documentation and manual pages, source headers, shared libraries and other resources such as graphic elements, sounds, fonts, document templates, translations and configuration files, or a combination of them.
Configuration files are usually the means through which the behaviour of software without a user interface is specified. This covers parts of the operating systems, network daemons and programs in general that don’t come with an interactive graphical or textual interface as the principal mean for setting options.
System wide configuration for operating system software tends to be kept under /etc, while configuration for software installed via pkgsrc ends up under LOCALBASE/etc (e.g., /usr/pkg/etc).
Software packaged as part of pkgsrc provides example configuration files, if any, which usually get extracted to LOCALBASE/share/examples/PKGBASE/.
Don’t worry: automatic merging is disabled by default, set $VCSAUTOMERGE to enable it.
In order to avoid breakage, installed configuration is backed up first in the VCS, separating user-modified files from files that have been already automatically merged in the past, in order to allow the administrator to easily restore the last manually edited file in case of breakage.
VCS functionality only applies to configuration files, not to rc.d scripts, and only if the environment variable $NOVCS is unset.
The version control system to be used as a backend can be set through $VCS. It default to RCS, the Revision Control System, which works only locally and doesn’t support atomic transactions.
Other backends such as CVS are supported and more will come; these, being used at the explicit request of the administrator, need to be already installed and placed in a directory part of $PATH.

pkgsrc is now able to deploy configuration from packages being installed from a remote, site-specific vcs repository.
User modified files are always tracked even if automerge functionality is not enabled, and a new tool, pkgconftrack(1), exists to manually store user changes made outside of package upgrade time.
Version Control software is executed as the same user running pkg_add or make install, unless the user is “root”. In this case, a separate, unprivileged user, pkgvcsconf, gets created with its own home directory and a working login shell (but no password). The home directory is not strictly necessary, it exists to facilitate migrations betweens repositories and vcs changes; it also serves to store keys used to access remote repositories.
Using git instead of rcs is simply done by setting VCS=git in pkg_install.conf

Support for configuration tracking is in scripts, pkginstall scripts, that get built into binary packages and are run by pkg_add upon installation. The idea behind the proposal suggested that users of the new feature should be able to store revisions of their installed configuration files, and of package-provided default, both in local or remote repositories. With this capability in place, it doesn’t take much to make the scripts “pull” configuration from a VCS repository at installation time.
That’s what setting VCSCONFPULL=yes in pkg_install.conf after having enabled VCSTRACK_CONF does: You are free to use official, third party prebuilt packages that have no customization in them, enable these options, and point pkgsrc to a private conf repository. If it contains custom configuration for the software you are installing, an attempt will be made to use it and install it on your system. If it fails, pkginstall will fall back to using the defaults that come inside the package. RC scripts are always deployed from the binary package, if existing and PKG_RCD_SCRIPTS=yes in pkg_install.conf or the environment.
This will be part of packages, not a separate solution like configuration management tools. It doesn’t support running scripts on the target system to customize the installation, it doesn’t come with its domain-specific language, it won’t run as a daemon or require remote logins to work. It’s quite limited in scope, but you can define a ROLE for your system in pkg_install.conf or in the environment, and pkgsrc will look for configuration you or your organization crafted for such a role (e.g., public, standalone webserver vs reverse proxy or node in a database cluster)

###A little bit of the one-time MacOS version still lingers in ZFS

Once upon a time, Apple came very close to releasing ZFS as part of MacOS. Apple did this work in its own copy of the ZFS source base (as far as I know), but the people in Sun knew about it and it turns out that even today there is one little lingering sign of this hoped-for and perhaps prepared-for ZFS port in the ZFS source code. Well, sort of, because it’s not quite in code.
Lurking in the function that reads ZFS directories to turn (ZFS) directory entries into the filesystem independent format that the kernel wants is the following comment:

objnum = ZFS_DIRENT_OBJ(zap.za_first_integer);
/*
* MacOS X can extract the object type here such as:
* uint8_t type = ZFS_DIRENT_TYPE(zap.za_first_integer);
*/

  • Specifically, this is in zfs_readdir in zfs_vnops.c .

ZFS maintains file type information in directories. This information can’t be used on Solaris (and thus Illumos), where the overall kernel doesn’t have this in its filesystem independent directory entry format, but it could have been on MacOS (‘Darwin’), because MacOS is among the Unixes that support d_type. The comment itself dates all the way back to this 2007 commit, which includes the change ‘reserve bits in directory entry for file type’, which created the whole setup for this.
I don’t know if this file type support was added specifically to help out Apple’s MacOS X port of ZFS, but it’s certainly possible, and in 2007 it seems likely that this port was at least on the minds of ZFS developers. It’s interesting but understandable that FreeBSD didn’t seem to have influenced them in the same way, at least as far as comments in the source code go; this file type support is equally useful for FreeBSD, and the FreeBSD ZFS port dates to 2007 too (per this announcement).
Regardless of the exact reason that ZFS picked up maintaining file type information in directory entries, it’s quite useful for people on both FreeBSD and Linux that it does so. File type information is useful for any number of things and ZFS filesystems can (and do) provide this information on those Unixes, which helps make ZFS feel like a truly first class filesystem, one that supports all of the expected general system features.

##Beastie Bits

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Sep 06 2018

1hr 13mins

Play

Rank #5: 326: Certified BSD

Podcast cover
Read more

LPI releases BSD Certification, openzfs trip report, Using FreeBSD with ports, LLDB threading support ready, Linux versus Open Source Unix, and more.

Headlines

Linux Professional Institute Releases BSD Specialist Certification - re BSD Certification Group

Linux Professional Institute extends its Open Technology certification track with the BSD Specialist Certification. Starting October 30, 2019, BSD Specialist exams will be globally available. The certification was developed in collaboration with the BSD Certification Group which merged with Linux Professional Institute in 2018.

G. Matthew Rice, the Executive Director of Linux Professional Institute says that "the release of the BSD Specialist certification marks a major milestone for Linux Professional Institute. With this new credential, we are reaffirming our belief in the value of, and support for, all open source technologies. As much as possible, future credentials and educational programs will include coverage of BSD.”

OpenZFS Trip Report

The seventh annual OpenZFS Developer Summit took place on November 4th and 5th in San Francisco and brought together a healthy mix of familiar faces and new community participants. Several folks from iXsystems took part in the talks, hacking, and socializing at this amazing annual event. The messages of the event can be summed up as Unification, Refinement, and Ecosystem Tooling.

News Roundup

Using FreeBSD with Ports (2/2): Tool-assisted updating

In the previous post I explained why sometimes building your software from ports may make sense on FreeBSD. I also introduced the reader to the old-fashioned way of using tools to make working with ports a bit more convenient.

In this follow-up post we’re going to take a closer look at portmaster and see how it especially makes updating from ports much, much easier. For people coming here without having read the previous article: What I describe here is not what every FreeBSD admin today should consider good practice (any more)! It can still be useful in special cases, but my main intention is to discuss this for building up the foundation for what you actually should do today.

LLDB Threading support now ready for mainline

Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages.

In February, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support, extending NetBSD's ptrace interface to cover more register types and fix compat32 issues and fixing watchpoint support. Then, I've started working on improving thread support which is taking longer than expected. You can read more about that in my September 2019 report.

So far the number of issues uncovered while enabling proper threading support has stopped me from merging the work-in-progress patches. However, I've finally reached the point where I believe that the current work can be merged and the remaining problems can be resolved afterwards. More on that and other LLVM-related events happening during the last month in this report.

Linux VS open source UNIX

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

Sponsored By:

Nov 28 2019

1hr

Play

Rank #6: 322: Happy Birthday, Unix

Podcast cover
Read more

Unix is 50, Hunting down Ken's PDP-7, OpenBSD and OPNSense have new releases, Clarification on what GhostBSD is, sshuttle - VPN over SSH, and more.

Headlines

Unix is 50

In the summer of 1969 computer scientists Ken Thompson and Dennis Ritchie created the first implementation of Unix with the goal of designing an elegant and economical operating system for a little-used PDP-7 minicomputer at Bell Labs. That modest project, however, would have a far-reaching legacy. Unix made large-scale networking of diverse computing systems — and the Internet — practical. The Unix team went on to develop the C language, which brought an unprecedented combination of efficiency and expressiveness to programming. Both made computing more "portable". Today, Linux, the most popular descendent of Unix, powers the vast majority of servers, and elements of Unix and Linux are found in most mobile devices. Meanwhile C++ remains one of the most widely used programming languages today. Unix may be a half-century old but its influence is only growing.

Hunting down Ken's PDP-7: video footage found

In my prior blog post, I traced Ken's scrounged PDP-7 to SN 34. In this post I'll show that we have actual video footage of that PDP-7 due to an old film from Bell Labs. this gives us almost a minute of footage of the PDP-7 Ken later used to create Unix.

News Roundup

OpenBSD 6.6 Released

OPNsense 19.7.5 released

Hello friends and followers, Lots of plugin and ports updates this time with a few minor improvements in all core areas. Behind the scenes we are starting to migrate the base system to version

12.1 which is supposed to hit the next 20.1 release. Stay tuned for more infos in the next month or so.

Here are the full patch notes:

  • system: show all swap partitions in system information widget
  • system: flatten services_get() in preparation for removal
  • system: pin Syslog-ng version to specific package name
  • system: fix LDAP/StartTLS with user import page
  • system: fix a PHP warning on authentication server page
  • system: replace most subprocess.call use
  • interfaces: fix devd handling of carp devices (contributed by stumbaumr)
  • firewall: improve firewall rules inline toggles
  • firewall: only allow TCP flags on TCP protocol
  • firewall: simplify help text for direction setting
  • firewall: make protocol log summary case insensitive
  • reporting: ignore malformed flow records
  • captive portal: fix type mismatch for timeout read
  • dhcp: add note for static lease limitation with lease registration (contributed by Northguy)
  • ipsec: add margintime and rekeyfuzz options
  • ipsec: clear $dpdline correctly if not set
  • ui: fix tokenizer reorder on multiple saves
  • plugins: os-acme-client 1.26[1]
  • plugins: os-bind will reload bind on record change (contributed by blablup)
  • plugins: os-etpro-telemetry minor subprocess.call replacement
  • plugins: os-freeradius 1.9.4[2]
  • plugins: os-frr 1.12[3]
  • plugins: os-haproxy 2.19[4]
  • plugins: os-mailtrail 1.2[5]
  • plugins: os-postfix 1.11[6]
  • plugins: os-rspamd 1.8[7]
  • plugins: os-sunnyvalley LibreSSL support (contributed by Sunny Valley Networks)
  • plugins: os-telegraf 1.7.6[8]
  • plugins: os-theme-cicada 1.21 (contributed by Team Rebellion)
  • plugins: os-theme-tukan 1.21 (contributed by Team Rebellion)
  • plugins: os-tinc minor subprocess.call replacement
  • plugins: os-tor 1.8 adds dormant mode disable option (contributed by Fabian Franz)
  • plugins: os-virtualbox 1.0 (contributed by andrewhotlab)

Dealing with the misunderstandings of what is GhostBSD

Since the release of 19.09, I have seen a lot of misunderstandings on what is GhostBSD and the future of GhostBSD. GhostBSD is based on TrueOS with FreeBSD 12 STABLE with our twist to it. We are still continuing to use TrueOS for OpenRC, and the new package's system for the base system that is built from ports. GhostBSD is becoming a slow-moving rolling release base on the latest TrueOS with FreeBSD 12 STABLE. When FreeBSD 13 STABLE gets released, GhostBSD will be upgraded to TrueOS with FreeBSD 13 STABLE.

Our official desktop is MATE, which means that the leading developer of GhostBSD does not officially support XFCE. Community releases are maintained by the community and for the community. GhostBSD project will provide help to build and to host the community release. If anyone wants to have a particular desktop supported, it is up to the community. Sure I will help where I can, answer questions and guide new community members that contribute to community release.

There is some effort going on for Plasma5 desktop. If anyone is interested in helping with XFCE and Plasma5 or in creating another community release, you are well come to contribute. Also, Contribution to the GhostBSD base system, to ports and new ports, and in house software are welcome. We are mostly active on Telegram https://t.me/ghostbsd, but you can also reach us on the forum.

SHUTTLE – VPN over SSH | VPN Alternative

Looking for a lightweight VPN client, but are not ready to spend a monthly recurring amount on a VPN? VPNs can be expensive depending upon the quality of service and amount of privacy you want. A good VPN plan can easily set you back by 10$ a month and even that doesn’t guarantee your privacy. There is no way to be sure whether the VPN is storing your confidential information and traffic logs or not. sshuttle is the answer to your problem it provides VPN over ssh and in this article we’re going to explore this cheap yet powerful alternative to the expensive VPNs. By using open source tools you can control your own privacy.

  • VPN over SSH – sshuttle

sshuttle is an awesome program that allows you to create a VPN connection from your local machine to any remote server that you have ssh access on. The tunnel established over the ssh connection can then be used to route all your traffic from client machine through the remote machine including all the dns traffic. In the bare bones sshuttle is just a proxy server which runs on the client machine and forwards all the traffic to a ssh tunnel. Since its open source it holds quite a lot of major advantages over traditional VPN.

OpenSSH 8.1 Released

  • Security

    • ssh(1), sshd(8), ssh-add(1), ssh-keygen(1): an exploitable integer overflow bug was found in the private key parsing code for the XMSS key type. This key type is still experimental and support for it is not compiled by default. No user-facing autoconf option exists in portable OpenSSH to enable it. This bug was found by Adam Zabrocki and reported via SecuriTeam's SSD program.
    • ssh(1), sshd(8), ssh-agent(1): add protection for private keys at rest in RAM against speculation and memory side-channel attacks like Spectre, Meltdown and Rambleed. This release encrypts private keys when they are not in use with a symmetric key that is derived from a relatively large "prekey" consisting of random data (currently 16KB).
  • This release includes a number of changes that may affect existing configurations:

    • ssh-keygen(1): when acting as a CA and signing certificates with an RSA key, default to using the rsa-sha2-512 signature algorithm. Certificates signed by RSA keys will therefore be incompatible with OpenSSH versions prior to 7.2 unless the default is overridden (using "ssh-keygen -t ssh-rsa -s ...").
  • New Features

    • ssh(1): Allow %n to be expanded in ProxyCommand strings
    • ssh(1), sshd(8): Allow prepending a list of algorithms to the default set by starting the list with the '' character, E.g. "HostKeyAlgorithms ssh-ed25519"
    • ssh-keygen(1): add an experimental lightweight signature and verification ability. Signatures may be made using regular ssh keys held on disk or stored in a ssh-agent and verified against an authorized_keys-like list of allowed keys. Signatures embed a namespace that prevents confusion and attacks between different usage domains (e.g. files vs email).
    • ssh-keygen(1): print key comment when extracting public key from a private key.
    • ssh-keygen(1): accept the verbose flag when searching for host keys in known hosts (i.e. "ssh-keygen -vF host") to print the matching host's random-art signature too.
    • All: support PKCS8 as an optional format for storage of private keys to disk. The OpenSSH native key format remains the default, but PKCS8 is a superior format to PEM if interoperability with non-OpenSSH software is required, as it may use a less insecure key derivation function than PEM's.

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

Oct 31 2019

1hr 7mins

Play

Rank #7: Episode 280: FOSS Clothing | BSD Now 280

Podcast cover
Read more

A EULA in FOSS clothing, NetBSD with more LLVM support, Thoughts on FreeBSD 12.0, FreeBSD Performance against Windows and Linux on Xeon, Microsoft shipping NetBSD, and more.

Headlines

A EULA in FOSS clothing?

There was a tremendous amount of reaction to and discussion about my blog entry on the midlife crisis in open source. As part of this discussion on HN, Jay Kreps of Confluent took the time to write a detailed response — which he shortly thereafter elevated into a blog entry.
Let me be clear that I hold Jay in high regard, as both a software engineer and an entrepreneur — and I appreciate the time he took to write a thoughtful response. That said, there are aspects of his response that I found troubling enough to closely re-read the Confluent Community License — and that in turn has led me to a deeply disturbing realization about what is potentially going on here.
To GitHub: Assuming that this is in fact a EULA, I think it is perilous to allow EULAs to sit in public repositories. It’s one thing to have one click through to accept a license (though again, that itself is dubious), but to say that a git clone is an implicit acceptance of a contract that happens to be sitting somewhere in the repository beggars belief. With efforts like choosealicense.com, GitHub has been a model in guiding projects with respect to licensing; it would be helpful for GitHub’s counsel to weigh in on their view of this new strain of source-available proprietary software and the degree to which it comes into conflict with GitHub’s own terms of service.
To foundations concerned with software liberties, including the Apache Foundation, the Linux Foundation, the Free Software Foundation, the Electronic Frontier Foundation, the Open Source Initiative, and the Software Freedom Conservancy: the open source community needs your legal review on this! I don’t think I’m being too alarmist when I say that this is potentially a dangerous new precedent being set; it would be very helpful to have your lawyers offer their perspectives on this, even if they disagree with one another. We seem to be in some terrible new era of frankenlicenses, where the worst of proprietary licenses are bolted on to the goodwill created by open source licenses; we need your legal voices before these creatures destroy the village!

NetBSD and LLVM

NetBSD entering 2019 with more complete LLVM support

I’m recently helping the NetBSD developers to improve the support for this operating system in various LLVM components. As you can read in my previous report, I’ve been focusing on fixing build and test failures for the purpose of improving the buildbot coverage.
Previously, I’ve resolved test failures in LLVM, Clang, LLD, libunwind, openmp and partially libc++. During the remainder of the month, I’ve been working on the remaining libc++ test failures, improving the NetBSD clang driver and helping Kamil Rytarowski with compiler-rt.

The process of upstreaming support to LLVM sanitizers has been finalized

I’ve finished the process of upstreaming patches to LLVM sanitizers (almost 2000LOC of local code) and submitted to upstream new improvements for the NetBSD support. Today out of the box (in unpatched version) we have support for a variety of compiler-rt LLVM features: ASan (finds unauthorized memory access), UBSan (finds unspecified code semantics), TSan (finds threading bugs), MSan (finds uninitialized memory use), SafeStack (double stack hardening), Profile (code coverage), XRay (dynamic code tracing); while other ones such as Scudo (hardened allocator) or DFSan (generic data flow sanitizer) are not far away from completeness.
The NetBSD support is no longer visibly lacking behind Linux in sanitizers, although there are still failing tests on NetBSD that are not observed on Linux. On the other hand there are features working on NetBSD that are not functional on Linux, like sanitizing programs during early initialization process of OS (this is caused by /proc dependency on Linux that is mounted by startup programs, while NetBSD relies on sysctl(3) interfaces that is always available).

News Roundup

Thoughts on FreeBSD 12.0

Playing with FreeBSD with past week I don’t feel as though there were any big surprises or changes in this release compared to FreeBSD 11. In typical FreeBSD fashion, progress tends to be evolutionary rather than revolutionary, and this release feels like a polished and improved incremental step forward. I like that the installer handles both UFS and ZFS guided partitioning now and in a friendly manner. In the past I had trouble getting FreeBSD’s boot menu to work with boot environments, but that has been fixed for this release.
I like the security options in the installer too. These are not new, but I think worth mentioning. FreeBSD, unlike most Linux distributions, offers several low-level security options (like hiding other users’ processes and randomizing PIDs) and I like having these presented at install time. It’s harder for people to attack what they cannot see, or predict, and FreeBSD optionally makes these little adjustment for us.
Something which stands out about FreeBSD, compared to most Linux distributions I run, is that FreeBSD rarely holds the user’s hand, but also rarely surprises the user. This means there is more reading to do up front and new users may struggle to get used to editing configuration files in a text editor. But FreeBSD rarely does anything unless told to do it. Updates rarely change the system’s behaviour, working technology rarely gets swapped out for something new, the system and its applications never crashed during my trial. Everything was rock solid. The operating system may seem like a minimal, blank slate to new users, but it’s wonderfully dependable and predictable in my experience.
I probably wouldn’t recommend FreeBSD for desktop use. It’s close relative, GhostBSD, ships with a friendly desktop and does special work to make end user applications run smoothly. But for people who want to run servers, possible for years without change or issues, FreeBSD is a great option. It’s also an attractive choice, in my opinion, for people who like to build their system from the ground up, like you would with Debian’s server install or Arch Linux. Apart from the base tools and documentation, there is nothing on a FreeBSD system apart from what we put on it.

FreeBSD 12.0 Performance Against Windows & Linux On An Intel Xeon Server

Last week I posted benchmarks of Windows Server 2019 against various Linux distributions using a Tyan dual socket Intel Xeon server. In this article are some complementary results when adding in the performance of FreeBSD 11.2 against the new FreeBSD 12.0 stable release for this leading BSD operating system. As some fun benchmarks to end out 2018, here are the results of FreeBSD 11.2/12.0 (including an additional run when using GCC rather than Clang) up against Windows Server and several enterprise-ready Linux distributions.
While FreeBSD 12.0 had picked up just one win of the Windows/Linux comparisons run, the FreeBSD performance is moving in the right direction. FreeBSD 12.0 was certainly faster than FreeBSD 11.2 on this dual Intel Xeon Scalable server based on a Tyan 1U platform. Meanwhile, to no surprise given the data last week, Clear Linux was by far the fastest out-of-the-box operating system tested.
I did run some extra benchmarks on FreeBSD 11.2/12.0 with this hardware: in total I ran 120 benchmarks for these BSD tests. Of the 120 tests, there were just 15 cases where FreeBSD 11.2 was faster than 12.0. Seeing FreeBSD 12.0 faster than 11.2 nearly 90% of the time is an accomplishment and usually with other operating systems we see more of a mixed bag on new releases with not such solidly better performance. It was also great seeing the competitive performance out of FreeBSD when using the Clang compiler for the source-based tests compared to the GCC8 performance. Additional data available via this OpenBenchmarking.org result file.

How NetBSD came to be shipped by Microsoft

Google cache in case the site is down

In 2000, Joe Britt, Matt Hershenson and Andy Rubin formed Danger Incorporated. Danger developed the world’s first recognizable smartphone, the Danger HipTop. T-Mobile sold the first HipTop under the brand name Sidekick in October of 2002.
Danger had a well developed kernel that had been designed and built in house. The kernel came to be viewed as not a core intellectual property and Danger started a search for a replacement. For business reasons, mostly to do with legal concerns over the Gnu Public License, Danger rejected Linux and began to consider BSD Unix as a replacement for the kernel.
In 2006 I was hired by Mike Chen, the manager of the kernel development group to investigate the feasibility of replacing the Danger kernel with a BSD kernel, to select the version of BSD to use, to develop a prototype and to develop the plan for adapting BSD to Danger’s requirements.
NetBSD was easily the best choice among the BSD variations at the time because it had well developed cross development tools. It was easy to use a NetBSD desktop running an Intel release to cross compile a NetBSD kernel and runtime for a device running an ARM processor. (Those interested in mailing list archaeology might be amused to investigate NetBSD technical mailing list for mail from picovex, particularly from Bucky Katz at picovex.)
We began product development on the specific prototype of the phone that would become the Sidekick LX2009 in 2007 and contracts for the phone were written with T-Mobile. We were about half way through the two year development cycle when Microsoft purchased Danger in 2008.
Microsoft would have preferred to ship the Sidekick running Windows/CE rather than NetBSD, but a schedule analysis performed by me, and another by an independent outside contractor, indicated that doing so would result in unacceptable delay.

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Jan 10 2019

52mins

Play

Rank #8: 321: The Robot OS

Podcast cover
Read more

An interview with Trenton Schulz about his early days with FreeBSD, Robot OS, Qt, and more.

Interview - Trenton Schulz - freenas@norwegianrockcat.com

Robot OS on FreeBSD

  • BR: Welcome to the show. Can you tell us a little bit about yourself and how you got started with BSD?
  • AJ: You were working for Trolltech (creators of Qt). Was FreeBSD used there and how?
  • BR: Can you tell us more about the work you are doing with Robot OS on FreeBSD?
  • AJ: Was EuroBSDcon your first BSD conference? How did you like it?
  • BR: Do you have some tips or advice on how to get started with the BSDs?
  • AJ: Is there anything else you’d like to tell us before we let you go?

Beastie Bits

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

Special Guest: Trenton Shulz.

Oct 24 2019

55mins

Play

Rank #9: 285: BSD Strategy

Podcast cover
Read more

Strategic thinking to keep FreeBSD relevant, reflecting on the soul of a new machine, 10GbE Benchmarks On Nine Linux Distros and FreeBSD, NetBSD integrating LLVM sanitizers in base, FreeNAS 11.2 distrowatch review, and more.

##Headlines
###Strategic thinking, or what I think what we need to do to keep FreeBSD relevant

Since I participate in the FreeBSD project there are from time to time some voices which say FreeBSD is dead, Linux is the way to go. Most of the time those voices are trolls, or people which do not really know what FreeBSD has to offer. Sometimes those voices wear blinders, they only see their own little world (were Linux just works fine) and do not see the big picture (like e.g. competition stimulates business, …) or even dare to look what FreeBSD has to offer.
Sometimes those voices raise a valid concern, and it is up to the FreeBSD project to filter out what would be beneficial. Recently there were some mails on the FreeBSD lists in the sense of “What about going into direction X?”. Some people just had the opinion that we should stay where we are. In my opinion this is similarly bad to blindly saying FreeBSD is dead and following the masses. It would mean stagnation. We should not hold people back in exploring new / different directions. Someone wants to write a kernel module in (a subset of) C++ or in Rust… well, go ahead, give it a try, we can put it into the Ports Collection and let people get experience with it.
This discussion on the mailinglists also triggered some kind of “where do we see us in the next years” / strategic thinking reflection. What I present here, is my very own opinion about things we in the FreeBSD project should look at, to stay relevant in the long term. To be able to put that into scope, I need to clarify what “relevant” means in this case.
FreeBSD is currently used by companies like Netflix, NetApp, Cisco, Juniper, and many others as a base for products or services. It is also used by end‐users as a work‐horse (e.g. mailservers, webservers, …). Staying relevant means in this context, to provide something which the user base is interested in to use and which makes it more easy / fast for the user base to deliver whatever they want or need to deliver than with another kind of system. And this in terms of time to market of a solution (time to deliver a service like a web‐/mail‐/whatever‐server or product), and in terms of performance (which not only means speed, but also security and reliability and …) of the solution.
I have categorized the list of items I think are important into (new) code/features, docs, polishing and project infrastructure. Links in the following usually point to documentation/HOWTOs/experiences for/with FreeBSD, and not to the canonical entry points of the projects or technologies. In a few cases the links point to an explanation in the wikipedia or to the website of the topic in question.

###Reflecting on The Soul of a New Machine

Long ago as an undergraduate, I found myself back home on a break from school, bored and with eyes wandering idly across a family bookshelf. At school, I had started to find a calling in computing systems, and now in the den, an old book suddenly caught my eye: Tracy Kidder’s The Soul of a New Machine. Taking it off the shelf, the book grabbed me from its first descriptions of Tom West, captivating me with the epic tale of the development of the Eagle at Data General. I — like so many before and after me — found the book to be life changing: by telling the stories of the people behind the machine, the book showed the creative passion among engineers that might otherwise appear anodyne, inspiring me to chart a course that might one day allow me to make a similar mark.
Since reading it over two decades ago, I have recommended The Soul of a Machine at essentially every opportunity, believing that it is a part of computing’s literary foundation — that it should be considered our Odyssey. Recently, I suggested it as beach reading to Jess Frazelle, and apparently with perfect timing: when I saw the book at the top of her vacation pile, I knew a fuse had been lit. I was delighted (though not at all surprised) to see Jess livetweet her admiration of the book, starting with the compelling prose, the lucid technical explanations and the visceral anecdotes — but then moving on to the deeper technical inspiration she found in the book. And as she reached the book’s crescendo, Jess felt its full power, causing her to reflect on the nature of engineering motivation.
Excited to see the effect of the book on Jess, I experienced a kind of reflected recommendation: I was inspired to (re-)read my own recommendation! Shortly after I started reading, I began to realize that (contrary to what I had been telling myself over the years!) I had not re-read the book in full since that first reading so many years ago. Rather, over the years I had merely revisited those sections that I remembered fondly. On the one hand, these sections are singular: the saga of engineers debugging a nasty I-cache data corruption issue; the young engineer who implements the simulator in an impossibly short amount of time because no one wanted to tell him that he was being impossibly ambitious; the engineer who, frustrated with a nanosecond-scale timing problem in the ALU that he designed, moved to a commune in Vermont, claiming a desire to deal with “no unit of time shorter than a season”. But by limiting myself to these passages, I was succumbing to the selection bias of my much younger self; re-reading the book now from start to finish has given new parts depth and meaning. Aspects that were more abstract to me as an undergraduate — from the organizational rivalries and absurdities of the industry to the complexities of West’s character and the tribulations of the team down the stretch — are now deeply evocative of concrete episodes of my own career.

  • See Article for rest…

##News Roundup

###Out-Of-The-Box 10GbE Network Benchmarks On Nine Linux Distributions Plus FreeBSD 12

Last week I started running some fresh 10GbE Linux networking performance benchmarks across a few different Linux distributions. That testing has now been extended to cover nine Linux distributions plus FreeBSD 12.0 to compare the out-of-the-box networking performance.
Tested this round alongside FreeBSD 12.0 was Antergos 19.1, CentOS 7, Clear Linux, Debian 9.6, Fedora Server 29, openSUSE Leap 15.0, openSUSE Tumbleweed, Ubuntu 18.04.1 LTS, and Ubuntu 18.10.
All of the tests were done with a Tyan S7106 1U server featuring two Intel Xeon Gold 6138 CPUs, 96GB of DDR4 system memory, and Samsung 970 EVO SSD. For the 10GbE connectivity on this server was an add-in HP NC523SFP PCIe adapter providing two 10Gb SPF+ ports using a QLogic 8214 controller.
Originally the plan as well was to include Windows Server 2016/2019. Unfortunately the QLogic driver download site was malfunctioning since Cavium’s acquisition of the company and the other Windows Server 2016 driver options not panning out and there not being a Windows Server 2019 option. So sadly that Windows testing was thwarted so I since started testing over with a Mellanox Connectx-2 10GbE NIC, which is well supported on Windows Server and so that testing is ongoing for the next article of Windows vs. Linux 10 Gigabit network performance plus some “tuned” Linux networking results too.

###Integration of the LLVM sanitizers with the NetBSD base system

Over the past month I’ve merged the LLVM compiler-rt sanitizers (LLVM svn r350590) with the base system. I’ve also managed to get a functional set of Makefile rules to build all of them, namely:
ASan, UBSan, TSan, MSan, libFuzzer, SafeStack, XRay.
In all supported variations and modes that are supported by the original LLVM compiler-rt package.

###Distrowatch FreeNAS 11.2 review

The project’s latest release is FreeNAS 11.2 and, at first, I nearly overlooked the new version because it appeared to be a minor point release. However, a lot of work went into the new version and 11.2 offers a lot of changes when compared next to 11.1, “including a major revamp of the web interface, support for self-encrypting drives, and new, backwards-compatible REST and WebSocket APIs. This update also introduces iocage for improved plugins and jails management and simplified plugin development.”

##Beastie Bits

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Feb 14 2019

1hr 9mins

Play

Rank #10: Episode 279: Future of ZFS | BSD Now 279

Podcast cover
Read more

The future of ZFS in FreeBSD, we pick highlights from the FreeBSD quarterly status report, flying with the raven, modern KDE on FreeBSD, many ways to launch FreeBSD in EC2, GOG installers on NetBSD, and more.

Headlines

The future of ZFS in FreeBSD

The sources for FreeBSD’s ZFS support are currently taken directly from Illumos with local ifdefs to support the peculiarities of FreeBSD where the Solaris Portability Layer (SPL) shims fall short. FreeBSD has regularly pulled changes from Illumos and tried to push back any bug fixes and new features done in the context of FreeBSD. In the past few years the vast majority of new development in ZFS has taken place in DelphixOS and zfsonlinux (ZoL). Earlier this year Delphix announced that they will be moving to ZoL: https://www.delphix.com/blog/kickoff-future-eko-2018 This shift means that there will be little to no net new development of Illumos. While working through the git history of ZoL I have also discovered that many races and locking bugs have been fixed in ZoL and never made it back to Illumos and thus FreeBSD. This state of affairs has led to a general agreement among the stakeholders that I have spoken to that it makes sense to rebase FreeBSD’s ZFS on ZoL. Brian Behlendorf has graciously encouraged me to add FreeBSD support directly to ZoL https://github.com/zfsonfreebsd/ZoF so that we might all have a single shared code base.
A port for ZoF can be found at https://github.com/miwi-fbsd/zof-port Before it can be committed some additional functionality needs to be added to the FreeBSD opencrypto framework. These can be found at https://reviews.freebsd.org/D18520
This port will provide FreeBSD users with multi modifier protection, project quotas, encrypted datasets, allocation classes, vectorized raidz, vectorized checksums, and various command line improvements.

FreeBSD Quarterly Status Update

With FreeBSD having gone all the way to 12, it is perhaps useful to take a look back at all the things that have been accomplished, in terms of many visible changes, as well as all the things that happen behind the scenes to ensure that FreeBSD continues to offer an alternative in both design, implementation, and execution.
The things you can look forward to reading about are too numerous to summarize, but cover just about everything from finalizing releases, administrative work, optimizations and depessimizations, features added and fixed, and many areas of improvement that might just surprise you a little.
Please have a cup of coffee, tea, hot cocoa, or other beverage of choice, and enjoy this culmulative set of reports covering everything that’s been done since October, 2017.
—Daniel Ebdrup

News Roundup

One year of flying with the Raven: Ready for the Desktop?

It has been a little over one year now that I’m with the Ravenports project. Time to reflect my involvement, my expectations and hopes.
  • Ravenports
Ravenports is a universal packaging framework for *nix operating systems. For the user it provides easy access to binary packages of common software for multiple platforms. It has been the long-lasting champion on Repology’s top 10 repositories regarding package freshness (rarely dropping below 96 percent while all other projects keep below 90!).
For the porter it offers a well-designed and elegant means of writing cross-platform buildsheets that allow building the same version of the software with (completely or mostly) the same compile-time configuration on different operating systems or distributions.
And for the developer it means a real-world project that’s written in modern Ada (ravenadm) and C (pkg) – as well as some Perl for support scripts and make. Things feel very optimized and fast. Not being a programmer though, I cannot really say anything about the actual code and thus leave it to the interested reader’s judgement.

Modern KDE on FreeBSD

New stuff in the official FreeBSD repositories! The X11 team has landed a newer version of libinput, opening up the way for KDE Plasma 5.14 in ports. That’s a pretty big update and it may frighten people with a new wallpaper.
What this means is that the graphical stack is once again on-par with what Plasma upstream expects, and we can get back to chasing releases as soon as they happen, rather than gnashing our teeth at missing dependencies. The KDE-FreeBSD CI servers are in the process of being upgraded to 12-STABLE, and we’re integrating with the new experimental CI systems as well. This means we are chasing sensibly-modern systems (13-CURRENT is out of scope).

The many ways to launch FreeBSD in EC2

Talking to FreeBSD users recently, I became aware that while I’ve created a lot of tools, I haven’t done a very good job of explaining how, and more importantly when to use them. So for all of the EC2-curious FreeBSD users out there: Here are the many ways to launch and configure FreeBSD in EC2 — ranging from the simplest to the most complicated (but most powerful):
  • Launch FreeBSD and SSH in
  • Launch FreeBSD and provide user-data
  • Use the AMI Builder to create a customized FreeBSD AMI
  • Build a FreeBSD AMI from a modified FreeBSD source tree
  • Build your own disk image
I hope I’ve provided tools which help you to run FreeBSD in EC2, no matter how common or unusual your needs are. If you find my work useful, please consider supporting my work in this area; while this is both something I enjoy working on and something which is useful for my day job (Tarsnap, my online backup service), having support would make it easier for me to prioritize FreeBSD/EC2 issues over other projects.

Using the GOG.com installers for Linux, on NetBSD

GOG.com prefers that you use their GOG Galaxy desktop app to download, install and manage all of your GOG games. But customers always have the option to install the game on their own terms, with a platform-specific installer.
GOG offers these installers for Mac, Windows and/or Linux, depending on which platforms the game is available for.
  • The installers truly are platform-specific:
  • macOS games are distributed in a standard .pkg
  • Windows games are distributed in a setup wizard .exe
  • Linux games are distributed in a goofy shell archive
Of course, none of those are NetBSD. So, if I wanted to even attempt to play a game distributed by GOG.com on NetBSD, which one should I pick? The obvious choice is the Linux installer, since Linux is the most similar to NetBSD, right? Au contraire! In practice, I found that it is easier to download the Windows installer.
Here’s what I mean. For example, I ported the open source version of Aquaria to pkgsrc, but that package is only the game’s engine, not the multimedia data. The multimedia data is still copyrighted. Therefore, you need to get it from somewhere else. GOG is usually a good choice, because they distribute their games without DRM. And as mentioned earlier, picking the Linux installer seemed like a natural choice.
Now, actually PLAYING the games on NetBSD is a separate matter entirely. The game I’ve got here, though, my current obsession Pyre, is built with MonoGame and therefore could theoretically work on NetBSD, too, with the help of a library called FNA and a script for OpenBSD called fnaify. I do hope to create a pkgsrc package for FNA and port the fnaify script to NetBSD at some point.

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Jan 03 2019

1hr 33mins

Play

Rank #11: 312: Why Package Managers

Podcast cover
Read more

The UNIX Philosophy in 2019, why use package managers, touchpad interrupted, Porting wine to amd64 on NetBSD second evaluation report, Enhancing Syzkaller Support for NetBSD, all about the Pinebook Pro, killing a process and all of its descendants, fast software the best software, and more.

Headlines

The UNIX Philosophy in 2019

Today, Linux and open source rules the world, and the UNIX philosophy is widely considered compulsory. Organizations are striving to build small, focused applications that work collaboratively in a cloud and microservices environment. We rely on the network, as well as HTTP (text) APIs for storing and referencing data. Moreover, nearly all configuration is stored and communicated using text (e.g. YAML, JSON or XML). And while the UNIX philosophy has changed dramatically over the past 5 decades, it hasn’t strayed too far from Ken Thompson’s original definition in 1973:

  • We write programs that do one thing and do it well
  • We write programs to work together
  • And we write programs that handle text streams, because that is a universal interface

Why Use Package Managers?

Valuable research is often hindered or outright prevented by the inability to install software. This need not be the case.

Since I began supporting research computing in 1999, I’ve frequently seen researchers struggle for days or weeks trying to install a single open source application. In most cases, they ultimately failed.

In many cases, they could have easily installed the software in seconds with one simple command, using a package manager such as Debian packages, FreeBSD ports, MacPorts, or Pkgsrc, just to name a few.

Developer websites often contain poorly written instructions for doing “caveman installs”; manually downloading, unpacking, patching, and building the software. The same laborious process must often be followed for other software packages on which it depends, which can sometimes number in the dozens. Many researchers are simply unaware that there are easier ways to install the software they need. Caveman installs are a colossal waste of man-hours. If 1000 people around the globe spend an average of 20 hours each trying to install the same program that could have been installed with a package manager (this is not uncommon), then 20,000 man-hours have been lost that could have gone toward science. How many important discoveries are delayed by this?

The elite research institutions have ample funding and dozens of IT staff dedicated to research computing. They can churn out publications even if their operation is inefficient. Most institutions, however, have few or no IT staff dedicated to research, and cannot afford to squander precious man-hours on temporary, one-off software installs. The wise approach for those of us in that situation is to collaborate on making software deployment easier for everyone. If we do so, then even the smallest research groups can leverage that work to be more productive and make more frequent contributions to science.

Fortunately, the vast majority of open source software installs can be made trivial for anyone to do for themselves. Modern package managers perform all the same steps as a caveman install, but automatically. Package managers also install dependencies for us automatically.

News Roundup

Touchpad, Interrupted

For two years I've been driving myself crazy trying to figure out the source of a driver problem on OpenBSD: interrupts never arrived for certain touchpad devices. A couple weeks ago, I put out a public plea asking for help in case any non-OpenBSD developers recognized the problem, but while debugging an unrelated issue over the weekend, I finally solved it.

It's been a long journey and it's a technical tale, but here it is.

Porting wine to amd64 on NetBSD, second evaluation report

  • Summary

Presently, Wine on amd64 is in test phase. It seems to work fine with caveats like LD_LIBRARY_PATH which has to be set as 32-bit Xorg libs don't have ${PREFIX}/emul/netbsd32/lib in its rpath section. The latter is due to us extracting 32-bit libs from tarballs in lieu of building 32-bit Xorg on amd64. As previously stated, pkgsrc doesn't search for pkgconfig files in ${PREFIX}/emul/netbsd32/lib which might have inadvertent effects that I am unaware of as of now. I shall be working on these issues during the final coding period. I would like to thank @leot, @maya and @christos for saving me from shooting myself in the foot many a time. I, admittedly, have had times when multiple approaches, which all seemed right at that time, perplexed me. I believe those are times when having a mentor counts, and I have been lucky enough to have really good ones. Once again, thanks to Google for this wonderful opportunity.

Enhancing Syzkaller Support for NetBSD, Part 2

As a part of Google Summer of Code’19, I am working on improving the support for Syzkaller kernel fuzzer. Syzkaller is an unsupervised coverage-guided kernel fuzzer, that supports a variety of operating systems including NetBSD. This report details the work done during the second coding period.

You can also take a look at the first report to learn more about the initial support that we added. : https://blog.netbsd.org/tnf/entry/enhancing_syzkaller_support_for_netbsd

July Update: All about the Pinebook Pro

"So I said I won’t be talking about the BSDs, but I feel like I should at the very least give you a general overview of the RK3399 *BSD functionality. I’ll make it quick. I’ve spoken to *BSD devs whom worked on the RockPro64 and from what I’ve gathered (despite the different *BSDs having varying degree of support for the RK3399 SOC) many of the core features are already supported, which bodes well for *BSD on the Pro. That said, some of the things you’d require on a functional laptop – such as the LCD (using eDP) for instance – will not work on the Pinebook Pro using *BSD as of today. So clearly a degree of work is yet needed for a BSD to run on the device. However, keep in mind that *BSD developers will be receiving their units soon and by the time you receive yours some basic functionality may be available."

Killing a process and all of its descendants

Killing processes in a Unix-like system can be trickier than expected. Last week I was debugging an odd issue related to job stopping on Semaphore. More specifically, an issue related to the killing of a running process in a job. Here are the highlights of what I learned:

Unix-like operating systems have sophisticated process relationships. Parent-child, process groups, sessions, and session leaders. However, the details are not uniform across operating systems like Linux and macOS. POSIX compliant operating systems support sending signals to process groups with a negative PID number.

Sending signals to all processes in a session is not trivial with syscalls.

Child processes started with exec inherit their parent signal configuration. If the parent process is ignoring the SIGHUP signal, for example, this configuration is propagated to the children.

The answer to the “What happens with orphaned process groups” question is not trivial.

Fast Software, the Best Software

I love fast software. That is, software speedy both in function and interface. Software with minimal to no lag between wanting to activate or manipulate something and the thing happening. Lightness.

Software that’s speedy usually means it’s focused. Like a good tool, it often means that it’s simple, but that’s not necessarily true. Speed in software is probably the most valuable, least valued asset. To me, speedy software is the difference between an application smoothly integrating into your life, and one called upon with great reluctance. Fastness in software is like great margins in a book — makes you smile without necessarily knowing why.

But why is slow bad? Fast software is not always good software, but slow software is rarely able to rise to greatness. Fast software gives the user a chance to “meld” with its toolset. That is, not break flow. When the nerds upon Nerd Hill fight to the death over Vi and Emacs, it’s partly because they have such a strong affinity for the flow of the application and its meldiness. They have invested. The Tool Is Good, so they feel. Not breaking flow is an axiom of great tools.

A typewriter is an excellent tool because, even though it’s slow in a relative sense, every aspect of the machine itself operates as quickly as the user can move. It is focused. There are no delays when making a new line or slamming a key into the paper. Yes, you have to put a new sheet of paper into the machine at the end of a page, but that action becomes part of the flow of using the machine, and the accumulation of paper a visual indication of work completed. It is not wasted work. There are no fundamental mechanical delays in using the machine. The best software inches ever closer to the physical directness of something like a typewriter. (The machine may break down, of course, ribbons need to be changed — but this is maintenance and separate from the use of the tool. I’d be delighted to “maintain” Photoshop if it would lighten it up.)

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

Aug 22 2019

1hr 12mins

Play

Rank #12: 306: Comparing Hammers

Podcast cover
Read more

Am5x86 based retro UNIX build log, setting up services in a FreeNAS Jail, first taste of DragonflyBSD, streaming Netflix on NetBSD, NetBSD on the last G4 Mac mini, Hammer vs Hammer2, and more.

Headlines

Polprog's Am5x86 based retro UNIX build log

I have recently acquired an Am5x86 computer, in a surprisingly good condition. This is an ongoing project, check this page often for updates!

I began by connecting a front panel. The panel came from a different chassis and is slightly too wide, so I had to attach it with a couple of zip-ties. However, that makes it stick out from the PC front at an angle, allowing easy access when the computer sits at the floor - and thats where it is most of the time. It's not that bad, to be honest, and its way easier to access than it would be, if mounted vertically

There is a mains switch on the front panel because the computer uses an older style power supply. Those power supplies instead of relying on a PSON signal, like modern ATX supplies, run a 4 wire cable to a mains switch. The cable carries live and neutral both ways, and the switch keys in or out the power. The system powers on as soon as the switch is enabled.

Originally there was no graphics card in it. Since a PC will not boot with out a GPU, I had to find one. The mainboard only has PCI and ISA slots, and all the GPUs I had were AGP. Fortunately, I bought a PCI GPU hoping it would solve my issue...

However the GPU turned out to be faulty. It took me some time to repair it. I had to repair a broken trace leading to one of the EEPROM pins, and replace a contact in the EEPROM's socket. Then I replaced all the electrolytic capacitors on it, and that fixed it for good.

Having used up only one of the three PCI slots, I populated the remaining pair with two ethernet cards. I still have a bunch of ISA slots available, but I have nothing to install there. Yet.

  • See the article for the rest of the writeup

Setting up services in a FreeNAS Jail

This piece demonstrates the setup of a server service in a FreeNAS jail and how to share files with a jail using Apache 2.4 as an example. Jails are powerful, self-contained FreeBSD environments with separate network settings, package management, and access to thousands of FreeBSD application packages. Popular packages such as Apache, NGINX, LigHTTPD, MySQL, and PHP can be found and installed with the pkg search and pkg install commands.

This example shows creating a jail, installing an Apache web server, and setting up a simple web page.

NOTE: Do not directly attach FreeNAS to an external network (WAN). Use port forwarding, proper firewalls and DDoS protections when using FreeNAS for external web sites. This example demonstrates expanding the functionality of FreeNAS in an isolated LAN environment.

News Roundup

First taste of DragonflyBSD

Last week, I needed to pick a BSD Operating System which supports NUMA to do some testing, so I decided to give Dragonfly BSD a shot. Dragonfly BSDonly can run on X86_64 architecture, which reminds me of Arch Linux, and after some tweaking, I feel Dragonfly BSD may be a “developer-friendly” Operating System, at least for me.

I mainly use Dragonfly BSD as a server, so I don’t care whether GUI is fancy or not. But I have high requirements of developer tools, i.e., compiler and debugger. The default compiler of Dragonfly BSD is gcc 8.3, and I can also install clang 8.0.0 from package. This means I can test state-of-the-art features of compilers, and it is really important for me. gdb‘s version is 7.6.1, a little lag behind, but still OK.

Furthermore, the upgradation of Dragonfly BSD is pretty simple and straightforward. I followed document to upgrade my Operating System to 5.6.0 this morning, just copied and pasted, no single error, booted successfully.

Streaming Netflix on NetBSD

Here's a step-by-step guide that allows streaming Netflix media on NetBSD using a intel-haxm accelerated QEMU vm.

Heads-up! Sound doesn't work, but everything else is fine. Please read the rest of this thread for a solution to this!!

“Sudo Mastery 2nd Edition” cover art reveal

I’m about halfway through the new edition of Sudo Mastery. Assuming nothing terrible happens, should have a complete first draft in four to six weeks. Enough stuff has changed in sudo that I need to carefully double-check every single feature. (I’m also horrified by the painfully obsolete versions of sudo shipped in the latest versions of CentOS and Debian, but people running those operating systems are already accustomed to their creaky obsolescence.)

But the reason for this blog post? I have Eddie Sharam’s glorious cover art. My Patronizers saw it last month, so now the rest of you get a turn.

NetBSD on the last G4 Mac mini

I'm a big fan of NetBSD. I've run it since 2000 on a Mac IIci (of course it's still running it) and I ran it for several years on a Power Mac 7300 with a G3 card which was the second incarnation of the Floodgap gopher server. Today I also still run it on a MIPS-based Cobalt RaQ 2 and an HP Jornada 690. I think NetBSD is a better match for smaller or underpowered systems than current-day Linux, and is fairly easy to harden and keep secure even though none of these systems are exposed to the outside world.

Recently I had a need to set up a bridge system that would be fast enough to connect two networks and I happened to have two of the "secret" last-of-the-line 1.5GHz G4 Mac minis sitting on the shelf doing nothing. Yes, they're probably outclassed by later Raspberry Pi models, but I don't have to buy anything and I like putting old hardware to good use.

Hammer vs Hammer2

With the newly released DragonFlyBSD 5.6 there are improvements to its original HAMMER2 file-system to the extent that it's now selected by its installer as the default file-system choice for new installations. Curious how the performance now compares between HAMMER and HAMMER2, here are some initial benchmarks on an NVMe solid-state drive using DragonFlyBSD 5.6.0.

With a 120GB Toshiba NVMe SSD on an Intel Core i7 8700K system, I ran some benchmarks of DragonFlyBSD 5.6.0 freshly installed with HAMMER2 and then again when returning to the original HAMMER file-system that remains available via its installer. No other changes were made to the setup during testing.

And then for the more synthetic workloads it was just a mix. But overall HAMMER2 was performing well during the initial testing and great to see it continuing to offer noticeable leads in real-world workloads compared to the aging HAMMER file-system. HAMMER2 also offers better clustering, online deduplication, snapshots, compression, encryption, and many other modern file-system features.

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

Jul 11 2019

38mins

Play

Rank #13: 296: It’s Alive: OpenBSD 6.5

Podcast cover
Read more

OpenBSD 6.5 has been released, mount ZFS datasets anywhere, help test upcoming NetBSD 9 branch, LibreSSL 2.9.1 is available, Bail Bond Denied Edition of FreeBSD Mastery: Jails, and one reason ed(1) was a good editor back in the days in this week’s episode.

Headlines

OpenBSD 6.5 Released

  • Changelog
  • Mirrors
  • 6.5 Includes
    • OpenSMTPD 6.5.0
    • LibreSSL 2.9.1
    • OpenSSH 8.0
    • Mandoc 1.14.5
    • Xenocara
    • LLVM/Clang 7.0.1 (+ patches)
    • GCC 4.2.1 (+ patches) and 3.3.6 (+ patches)
  • Many pre-built packages for each architecture:
    • aarch64: 9654
    • amd64: 10602
    • i386: 10535

Mount your ZFS datasets anywhere you want

ZFS is very flexible about mountpoints, and there are many features available to provide great flexibility.
When you create zpool maintank, the default mountpoint is /maintank.
You might be happy with that, but you don’t have to be content. You can do magical things.

  • Some highlights are:
    • mount point can be inherited
    • not all filesystems in a zpool need to be mounted
    • each filesystem (directory) can have different ZFS characteristics
    • In my case, let’s look at this new zpool I created earlier today and I will show you some very simple alternatives. This zpool use NVMe devices which should be faster than SSDs especially when used with multiple concurrent writes. This is my plan: run all the Bacula regression tests concurrently.

News Roundup

Branch for netbsd 9 upcoming, please help and test -current

Folks,
once again we are quite late for branching the next NetBSD release (NetBSD 9).
Initially planned to happen early in February 2019, we are now approaching May and it is unlikely that the branch will happen before that.
On the positive side, lots of good things landed in -current in between, like new Mesa, new jemalloc, lots of ZFS improvements - and some of those would be hard to pull up to the branch later.
On the bad side we saw lots of churn in -current recently, and there is quite some fallout where we not even have a good overview right now. And this is where you can help:

  • please test -current, on all the various machines you have
  • especially interesting would be test results from uncommon architectures
    or strange combinations (like the sparc userland on sparc64 kernel issue
    I ran in yesterday)
    Please test, report success, and file PRs for failures!
    We will likely announce the real branch date on quite short notice, the likely next candidates would be mid may or end of may.
    We may need to do extra steps after the branch (like switch some architectures back to old jemalloc on the branch). However, the less difference between -current and the branch, the easier will the release cycle go.
    Our goal is to have an unprecedented short release cycle this time. But..
    we always say that upfront.

LibreSSL 2.9.1 Released

We have released LibreSSL 2.9.1, which will be arriving in the LibreSSL
directory of your local OpenBSD mirror soon. This is the first stable release
from the 2.9 series, which is also included with OpenBSD 6.5

It includes the following changes and improvements from LibreSSL 2.8.x:

  • API and Documentation Enhancements

    • CRYPTO_LOCK is now automatically initialized, with the legacy
      callbacks stubbed for compatibility.
    • Added the SM3 hash function from the Chinese standard GB/T 32905-2016.
    • Added the SM4 block cipher from the Chinese standard GB/T 32907-2016.
    • Added more OPENSSLNO* macros for compatibility with OpenSSL.
    • Partial port of the OpenSSL ECKEYMETHOD API for use by OpenSSH.
    • Implemented further missing OpenSSL 1.1 API.
    • Added support for XChaCha20 and XChaCha20-Poly1305.
    • Added support for AES key wrap constructions via the EVP interface.
  • Compatibility Changes

    • Added pbkdf2 key derivation support to openssl(1) enc.
    • Changed the default digest type of openssl(1) enc to sha256.
    • Changed the default digest type of openssl(1) dgst to sha256.
    • Changed the default digest type of openssl(1) x509 -fingerprint to sha256.
    • Changed the default digest type of openssl(1) crl -fingerprint to sha256.
  • Testing and Proactive Security

    • Added extensive interoperability tests between LibreSSL and OpenSSL
      1.0 and 1.1.
    • Added additional Wycheproof tests and related bug fixes.
  • Internal Improvements

    • Simplified sigalgs option processing and handshake signing
      algorithm selection.
    • Added the ability to use the RSA PSS algorithm for handshake signatures.
    • Added bnrandinterval() and use it in code needing ranges of
      random bn values.
    • Added functionality to derive early, handshake, and application
      secrets as per RFC8446.
    • Added handshake state machine from RFC8446.
    • Removed some ASN.1 related code from libcrypto that had not been
      used since around 2000.
    • Unexported internal symbols and internalized more record layer structs.
    • Removed SHA224 based handshake signatures from consideration for
      use in a TLS 1.2 handshake.
  • Portable Improvements

    • Added support for assembly optimizations on 32-bit ARM ELF targets.
    • Added support for assembly optimizations on Mingw-w64 targets.
    • Improved Android compatibility
  • Bug Fixes

    • Improved protection against timing side channels in ECDSA signature
      generation.
    • Coordinate blinding was added to some elliptic curves. This is the
      last bit of the work by Brumley et al. to protect against the Portsmash
      vulnerability.
    • Ensure transcript handshake is always freed with TLS 1.2.

The LibreSSL project continues improvement of the codebase to reflect modern,
safe programming practices. We welcome feedback and improvements from the
broader community. Thanks to all of the contributors who helped make this
release possible.

FreeBSD Mastery: Jails – Bail Bond Denied Edition

I had a brilliant, hideous idea: to produce a charity edition of FreeBSD Mastery: Jails featuring the cover art I would use if I was imprisoned and did not have access to a real cover artist. (Never mind that I wouldn’t be permitted to release books while in jail: we creative sorts scoff at mere legal and cultural details.)
I originally wanted to produce my own take on the book’s cover art. My first attempt failed spectacularly.
I downgraded my expectations and tried again. And again. And again.
I’m pleased to reveal the final cover for FreeBSD Mastery: Jails–Bail Bond Edition!
This cover represents the very pinnacle of my artistic talents, and is the result of literally hours of effort.
But, as this book is available only to the winner of charity fund-raisers, purchase of this tome represents moral supremacy. I recommend flaunting it to your family, coworkers, and all those of lesser character.
Get your copy by winning the BSDCan 2019 charity auction… or any other other auction-type event I deem worthwhile.
As far as my moral fiber goes: I have learned that art is hard, and that artists are not paid enough.
And if I am ever imprisoned, I do hope that you’ll contribute to my bail fund. Otherwise, you’ll get more covers like this one.

One reason ed(1) was a good editor back in the days of V7 Unix

It is common to describe ed(1) as being line oriented, as opposed to screen oriented editors like vi. This is completely accurate but it is perhaps not a complete enough description for today, because ed is line oriented in a way that is now uncommon. After all, you could say that your shell is line oriented too, and very few people use shells that work and feel the same way ed does.
The surface difference between most people's shells and ed is that most people's shells have some version of cursor based interactive editing. The deeper difference is that this requires the shell to run in character by character TTY input mode, also called raw mode. By contrast, ed runs in what Unix usually calls cooked mode, where it reads whole lines from the kernel and the kernel handles things like backspace. All of ed's commands are designed so that they work in this line focused way (including being terminated by the end of the line), and as a whole ed's interface makes this whole line input approach natural. In fact I think ed makes it so natural that it's hard to think of things as being any other way. Ed was designed for line at a time input, not just to not be screen oriented.
This input mode difference is not very important today, but in the days of V7 and serial terminals it made a real difference. In cooked mode, V7 ran very little code when you entered each character; almost everything was deferred until it could be processed in bulk by the kernel, and then handed to ed all in a single line which ed could also process all at once. A version of ed that tried to work in raw mode would have been much more resource intensive, even if it still operated on single lines at a time.

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv


Your browser does not support the HTML5 video tag.

May 03 2019

1hr 1min

Play

Rank #14: Episode 265: Software Disenchantment | BSD Now 265

Podcast cover
Read more

We report from our experiences at EuroBSDcon, disenchant software, LLVM 7.0.0 has been released, Thinkpad BIOS update options, HardenedBSD Foundation announced, and ZFS send vs. rsync.

##Headlines

###[FreeBSD DevSummit & EuroBSDcon 2018 in Romania]

  • Your hosts are back from EuroBSDcon 2018 held in Bucharest, Romania this year. The first two days of the conference are used for tutorials and devsummits (FreeBSD and NetBSD), while the last two are for talks.
  • Although Benedict organized the devsummit in large parts, he did not attend it this year. He held his Ansible tutorial in the morning of the first day, followed by Niclas Zeising’s new ports and poudriere tutorial (which had a record attendance). It was intended for beginners that had never used poudriere before and those who wanted to create their first port. The tutorial was well received and Niclas already has ideas for extending it for future conferences.
  • On the second day, Benedict took Kirk McKusick’s “An Introduction to the FreeBSD Open-Source Operating System” tutorial, held as a one full day class this year. Although it was reduced in content, it went into enough depth of many areas of the kernel and operating system to spark many questions from attendees. Clearly, this is a good start into kernel programming as Kirk provides enough material and backstories to understand why certain things are implemented as they are.
  • Olivier Robert took [https://www.talegraph.com/tales/l2o9ltrvsE](pictures from the devsummit) and created a nice gallery out of it.
  • Devsummit evenings saw dinners at two restaurants that allowed developers to spend some time talking over food and drinks.
  • The conference opened on the next day with the opening session held by Mihai Carabas. He introduced the first keynote speaker, a colleague of his who presented “Lightweight virtualization with LightVM and Unikraft”.
  • Benedict helped out at the FreeBSD Foundation sponsor table and talked to people. He saw the following talks in between:

Selfhosting as an alternative to the public cloud (by Albert Dengg)
Using Boot Environments at Scale (by Allan Jude)
Livepatching FreeBSD kernel (by Maciej Grochowski)
FreeBSD: What to (Not) Monitor (by Andrew Fengler)
FreeBSD Graphics (by Niclas Zeising)

  • Allan spent a lot of time talking to people and helping track down issues they were having, in addition to attending many talks:

    Hacking together a FreeBSD presentation streaming box – For as little as possible (by Tom Jones)
    Introduction of FreeBSD in new environments (by Baptiste Daroussin)
    Keynote: Some computing and networking historical perspectives (by Ron Broersma)
    Livepatching FreeBSD kernel (by Maciej Grochowski)
    FreeBSD: What to (Not) Monitor (by Andrew Fengler)
    Being a BSD user (by Roller Angel)
    From “Hello World” to the VFS Layer: building a beadm for DragonFly BSD (by Michael Voight)

  • We also met the winner of our Power Bagel raffle from Episode 2^8. He received the item in the meantime and had it with him at the conference, providing a power outlet to charge other people’s devices.
  • During the closing session, GroffTheBSDGoat was handed over to Deb Goodkin, who will bring the little guy to the Grace Hopper Celebration of Women in Computing conference and then to MeetBSD later this year. It was also revealed that next year’s EuroBSDcon will be held in Lillehammer, Norway.
  • Thanks to all the speakers, helpers, sponsors, organizers, and attendees for making it a successful conferences. There were no talks recorded this year, but the slides will be uploaded to the EuroBSDcon website in a couple of weeks. The OpenBSD talks are already available, so check them out.

###Software disenchantment

I’ve been programming for 15 years now. Recently our industry’s lack of care for efficiency, simplicity, and excellence started really getting to me, to the point of me getting depressed by my own career and the IT in general.
Modern cars work, let’s say for the sake of argument, at 98% of what’s physically possible with the current engine design. Modern buildings use just enough material to fulfill their function and stay safe under the given conditions. All planes converged to the optimal size/form/load and basically look the same.
Only in software, it’s fine if a program runs at 1% or even 0.01% of the possible performance. Everybody just seems to be ok with it. People are often even proud about how much inefficient it is, as in “why should we worry, computers are fast enough”:
@tveastman: I have a Python program I run every day, it takes 1.5 seconds. I spent six hours re-writing it in rust, now it takes 0.06 seconds. That efficiency improvement means I’ll make my time back in 41 years, 24 days :-)
You’ve probably heard this mantra: “programmer time is more expensive than computer time”. What it means basically is that we’re wasting computers at an unprecedented scale. Would you buy a car if it eats 100 liters per 100 kilometers? How about 1000 liters? With computers, we do that all the time.

  • Everything is unbearably slow

Look around: our portable computers are thousands of times more powerful than the ones that brought man to the moon. Yet every other webpage struggles to maintain a smooth 60fps scroll on the latest top-of-the-line MacBook Pro. I can comfortably play games, watch 4K videos but not scroll web pages? How is it ok?
Google Inbox, a web app written by Google, running in Chrome browser also by Google, takes 13 seconds to open moderately-sized emails:
It also animates empty white boxes instead of showing their content because it’s the only way anything can be animated on a webpage with decent performance. No, decent doesn’t mean 60fps, it’s rather “as fast as this web page could possibly go”. I’m dying to see web community answer when 120Hz displays become mainstream. Shit barely hits 60Hz already.
Windows 10 takes 30 minutes to update. What could it possibly be doing for that long? That much time is enough to fully format my SSD drive, download a fresh build and install it like 5 times in a row.
Pavel Fatin: Typing in editor is a relatively simple process, so even 286 PCs were able to provide a rather fluid typing experience.
Modern text editors have higher latency than 42-year-old Emacs. Text editors! What can be simpler? On each keystroke, all you have to do is update tiny rectangular region and modern text editors can’t do that in 16ms. It’s a lot of time. A LOT. A 3D game can fill the whole screen with hundreds of thousands (!!!) of polygons in the same 16ms and also process input, recalculate the world and dynamically load/unload resources. How come?
As a general trend, we’re not getting faster software with more features. We’re getting faster hardware that runs slower software with the same features. Everything works way below the possible speed. Ever wonder why your phone needs 30 to 60 seconds to boot? Why can’t it boot, say, in one second? There are no physical limitations to that. I would love to see that. I would love to see limits reached and explored, utilizing every last bit of performance we can get for something meaningful in a meaningful way.

  • Everything is HUUUUGE

And then there’s bloat. Web apps could open up to 10× faster if you just simply block all ads. Google begs everyone to stop shooting themselves in their feet with AMP initiative—a technology solution to a problem that doesn’t need any technology, just a little bit of common sense. If you remove bloat, the web becomes crazy fast. How smart do you have to be to understand that?
Android system with no apps takes almost 6 Gb. Just think for a second how obscenely HUGE that number is. What’s in there, HD movies? I guess it’s basically code: kernel, drivers. Some string and resources too, sure, but those can’t be big. So, how many drivers do you need for a phone?
Windows 95 was 30Mb. Today we have web pages heavier than that! Windows 10 is 4Gb, which is 133 times as big. But is it 133 times as superior? I mean, functionally they are basically the same. Yes, we have Cortana, but I doubt it takes 3970 Mb. But whatever Windows 10 is, is Android really 150% of that?
Google keyboard app routinely eats 150 Mb. Is an app that draws 30 keys on a screen really five times more complex than the whole Windows 95? Google app, which is basically just a package for Google Web Search, is 350 Mb! Google Play Services, which I do not use (I don’t buy books, music or videos there)—300 Mb that just sit there and which I’m unable to delete.
All that leaves me around 1 Gb for my photos after I install all the essential (social, chats, maps, taxi, banks etc) apps. And that’s with no games and no music at all! Remember times when an OS, apps and all your data fit on a floppy?
Your desktop todo app is probably written in Electron and thus has userland driver for Xbox 360 controller in it, can render 3d graphics and play audio and take photos with your web camera.
A simple text chat is notorious for its load speed and memory consumption. Yes, you really have to count Slack in as a resource-heavy application. I mean, chatroom and barebones text editor, those are supposed to be two of the less demanding apps in the whole world. Welcome to 2018.
At least it works, you might say. Well, bigger doesn’t imply better. Bigger means someone has lost control. Bigger means we don’t know what’s going on. Bigger means complexity tax, performance tax, reliability tax. This is not the norm and should not become the norm. Overweight apps should mean a red flag. They should mean run away scared.

  • Better world manifesto

I want to see progress. I want change. I want state-of-the-art in software engineering to improve, not just stand still. I don’t want to reinvent the same stuff over and over, less performant and more bloated each time. I want something to believe in, a worthy end goal, a future better than what we have today, and I want a community of engineers who share that vision.
What we have today is not progress. We barely meet business goals with poor tools applied over the top. We’re stuck in local optima and nobody wants to move out. It’s not even a good place, it’s bloated and inefficient. We just somehow got used to it.
So I want to call it out: where we are today is bullshit. As engineers, we can, and should, and will do better. We can have better tools, we can build better apps, faster, more predictable, more reliable, using fewer resources (orders of magnitude fewer!). We need to understand deeply what are we doing and why. We need to deliver: reliably, predictably, with topmost quality. We can—and should–take pride in our work. Not just “given what we had…”—no buts!
I hope I’m not alone at this. I hope there are people out there who want to do the same. I’d appreciate if we at least start talking about how absurdly bad our current situation in the software industry is. And then we maybe figure out how to get out.

##News Roundup
###[llvm-announce] LLVM 7.0.0 Release

I am pleased to announce that LLVM 7 is now available. Get it here: https://llvm.org/releases/download.html#7.0.0 The release contains the work on trunk up to SVN revision 338536 plus work on the release branch. It is the result of the community's work over the past six months, including: function multiversioning in Clang with the 'target' attribute for ELF-based x86/x86_64 targets, improved PCH support in clang-cl, preliminary DWARF v5 support, basic support for OpenMP 4.5 offloading to NVPTX, OpenCL C++ support, MSan, X-Ray and libFuzzer support for FreeBSD, early UBSan, X-Ray and libFuzzer support for OpenBSD, UBSan checks for implicit conversions, many long-tail compatibility issues fixed in lld which is now production ready for ELF, COFF and MinGW, new tools llvm-exegesis, llvm-mca and diagtool. And as usual, many optimizations, improved diagnostics, and bug fixes. For more details, see the release notes: https://llvm.org/releases/7.0.0/docs/ReleaseNotes.html https://llvm.org/releases/7.0.0/tools/clang/docs/ReleaseNotes.html https://llvm.org/releases/7.0.0/tools/clang/tools/extra/docs/ReleaseNotes.html https://llvm.org/releases/7.0.0/tools/lld/docs/ReleaseNotes.html Thanks to everyone who helped with filing, fixing, and code reviewing for the release-blocking bugs! Special thanks to the release testers and packagers: Bero Rosenkränzer, Brian Cain, Dimitry Andric, Jonas Hahnfeld, Lei Huang Michał Górny, Sylvestre Ledru, Takumi Nakamura, and Vedant Kumar. For questions or comments about the release, please contact the community on the mailing lists. Onwards to LLVM 8! Cheers, Hans

###Update your Thinkpad’s bios with Linux or OpenBSD

  • Get your new bios

At first, go to the Lenovo website and download your new bios:

  • Go to lenovo support
  • Use the search bar to find your product (example for me, x270)
  • Choose the right product (if necessary) and click search
  • On the right side, click on Update Your System
  • Click on BIOS/UEFI
  • Choose *BIOS Update (Bootable CD) for Windows *
  • Download

For me the file is called like this : r0iuj25wd.iso

  • Extract bios update

Now you will need to install geteltorito.

  • With OpenBSD:

$ doas pkg_add geteltorito
quirks-3.7 signed on 2018-09-09T13:15:19Z
geteltorito-0.6: ok

  • With Debian:

$ sudo apt-get install genisoimage

  • Now we will extract the bios update :

$ geteltorito -o bios_update.img r0iuj25wd.iso
Booting catalog starts at sector: 20
Manufacturer of CD: NERO BURNING ROM VER 12
Image architecture: x86
Boot media type is: harddisk
El Torito image starts at sector 27 and has 43008 sector(s) of 512 Bytes

Image has been written to file "bios_update.img".
This will create a file called bios_update.img.

  • Put the image on an USB key
  • CAREFULL : on my computer, my USB key is sda1 on Linux and sd1 on OpenBSD.

Please check twice on your computer the name of your USB key.

  • With OpenBSD :

$ doas dd if=bios_update.img of=/dev/rsd1c

  • With Linux :

$ sudo dd if=bios_update.img of=/dev/sda

Now all you need is to reboot, to boot on your USB key and follow the instructions. Enjoy 😉

###Announcing The HardenedBSD Foundation

In June of 2018, we announced our intent to become a not-for-profit, tax-exempt 501©(3) organization in the United States. It took a dedicated team months of work behind-the-scenes to make that happen. On 06 September 2018, HardenedBSD Foundation Corp was granted 501©(3) status, from which point all US-based persons making donations can deduct the donation from their taxes.
We are grateful for those who contribute to HardenedBSD in whatever way they can. Thank you for making HardenedBSD possible. We look forward to a bright future, driven by a helpful and positive community.

###How you migrate ZFS filesystems matters

If you want to move a ZFS filesystem around from one host to another, you have two general approaches; you can use ‘zfs send’ and ‘zfs receive’, or you can use a user level copying tool such as rsync (or ‘tar -cf | tar -xf’, or any number of similar options). Until recently, I had considered these two approaches to be more or less equivalent apart from their convenience and speed (which generally tilted in favour of ‘zfs send’). It turns out that this is not necessarily the case and there are situations where you will want one instead of the other.
We have had two generations of ZFS fileservers so far, the Solaris ones and the OmniOS ones. When we moved from the first generation to the second generation, we migrated filesystems across using ‘zfs send’, including the filesystem with my home directory in it (we did this for various reasons). Recently I discovered that some old things in my filesystem didn’t have file type information in their directory entries. ZFS has been adding file type information to directories for a long time, but not quite as long as my home directory has been on ZFS.
This illustrates an important difference between the ‘zfs send’ approach and the rsync approach, which is that zfs send doesn’t update or change at least some ZFS on-disk data structures, in the way that re-writing them from scratch from user level does. There are both positives and negatives to this, and a certain amount of rewriting does happen even in the ‘zfs send’ case (for example, all of the block pointers get changed, and ZFS will re-compress your data as applicable).
I knew that in theory you had to copy things at the user level if you wanted to make sure that your ZFS filesystem and everything in it was fully up to date with the latest ZFS features. But I didn’t expect to hit a situation where it mattered in practice until, well, I did. Now I suspect that old files on our old filesystems may be partially missing a number of things, and I’m wondering how much of the various changes in ‘zfs upgrade -v’ apply even to old data.
(I’d run into this sort of general thing before when I looked into ext3 to ext4 conversion on Linux.)
With all that said, I doubt this will change our plans for migrating our ZFS filesystems in the future (to our third generation fileservers). ZFS sending and receiving is just too convenient, too fast and too reliable to give up. Rsync isn’t bad, but it’s not the same, and so we only use it when we have to (when we’re moving only some of the people in a filesystem instead of all of them, for example).
PS: I was going to try to say something about what ‘zfs send’ did and didn’t update, but having looked briefly at the code I’ve concluded that I need to do more research before running my keyboard off. In the mean time, you can read the OpenZFS wiki page on ZFS send and receive, which has plenty of juicy technical details.
PPS: Since eliminating all-zero blocks is a form of compression, you can turn zero-filled files into sparse files through a ZFS send/receive if the destination has compression enabled. As far as I know, genuine sparse files on the source will stay sparse through a ZFS send/receive even if they’re sent to a destination with compression off.

##Beastie Bits

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Sep 27 2018

1hr 41mins

Play

Rank #15: 289: Microkernel Failure

Podcast cover
Read more

A kernel of failure, IPv6 fragmentation vulnerability in OpenBSD’s pf, a guide to the terminal, using a Yubikey for SSH public key authentication, FreeBSD desktop series, and more.

##Headlines

###A Kernel Of Failure -
How IBM bet big on the microkernel being the next big thing in operating systems back in the ’90s—and spent billions with little to show for it.

Today in Tedium: In the early 1990s, we had no idea where the computer industry was going, what the next generation would look like, or even what the driving factor would be. All the developers back then knew is that the operating systems available in server rooms or on desktop computers simply weren’t good enough, and that the next generation needed to be better—a lot better. This was easier said than done, but this problem for some reason seemed to rack the brains of one company more than any other: IBM. Throughout the decade, the company was associated with more overwrought thinking about operating systems than any other, with little to show for it in the end. The problem? It might have gotten caught up in kernel madness. Today’s Tedium explains IBM’s odd operating system fixation, and the belly flops it created.

###CVE-2019-5597IPv6 fragmentation vulnerability in OpenBSD Packet Filter

Packet Filter is OpenBSD’s service for filtering network traffic and performing Network Address Translation. Packet Filter is also capable of normalizing and conditioning TCP/IP traffic, as well as providing bandwidth control and packet prioritization.
Packet Filter has been a part of the GENERIC kernel since OpenBSD 5.0.Because other BSD variants import part of OpenBSD code, Packet Filter is also shipped with at least the following distributions that are affected in a lesser extent: FreeBSD, pfSense, OPNSense, Solaris.

Note that other distributions may also contain Packet Filter but due to the imported version they might not be vulnerable. This advisory covers the latest OpenBSD’s Packet Filter. For specific details about other distributions, please refer to the advisory of the affected product.

##News Roundup
###How I’m still not using GUIs in 2019: A guide to the terminal

TL;DR: Here are my dotfiles. Use them and have fun.

GUIs are bloatware. I’ve said it before. However, rather than just complaining about IDEs I’d like to provide an understandable guide to a much better alternative: the terminal.
IDE stands for Integrated Development Environment. This might be an accurate term, but when it comes to a real integrated development environment, the terminal is a lot better.
In this post, I’ll walk you through everything you need to start making your terminal a complete development environment: how to edit text efficiently, configure its appearance, run and combine a myriad of programs, and dynamically create, resize and close tabs and windows.

  • Don’t forget rule number one.

Whenever in doubt, read the manual.

###Using a Yubikey as smartcard for SSH public key authentication

SSH is an awesome tool. Logging into other machines securely is so pervasive to us sysadmins nowadays that few of us think about what’s going on underneath. Even more so once you start using the more advanced features such as the ssh-agent, agent-forwarding and ProxyJump. When doing so, care must be taken in order to not compromise one’s logins or ssh keys.
You might have heard of Yubikeys.
These are USB authentication devices that support several different modes: they can be used for OTP (One Time Password) authentication, they can store OpenPGP keys, be a 2-factor authentication token and they can act as a SmartCard.
In OpenBSD, you can use them for Login (with login_yubikey(8)) with OTP since 2012, and there are many descriptions available(1) how to set this up.

###The 18 Part FreeBSD Desktop Series by Vermaden

##Beastie Bits

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

Mar 14 2019

1hr 1min

Play

Rank #16: 302: Contention Reduction

Podcast cover
Read more

DragonFlyBSD's kernel optimizations pay off, differences between OpenBSD and Linux, NetBSD 2019 Google Summer of Code project list, Reducing that contention, fnaify 1.3 released, vmctl(8): CLI syntax changes, and things that Linux distributions should not do when packaging.

Headlines

DragonFlyBSD's Kernel Optimizations Are Paying Off

DragonFlyBSD lead developer Matthew Dillon has been working on a big VM rework in the name of performance and other kernel improvements recently. Here is a look at how those DragonFlyBSD 5.5-DEVELOPMENT improvements are paying off compared to DragonFlyBSD 5.4 as well as FreeBSD 12 and five Linux distribution releases. With Dillon using an AMD Ryzen Threadripper system, we used that too for this round of BSD vs. Linux performance benchmarks.
The work by Dillon on the VM overhaul and other changes (including more HAMMER2 file-system work) will ultimately culminate with the DragonFlyBSD 5.6 release (well, unless he opts for DragonFlyBSD 6.0 or so). These are benchmarks of the latest DragonFlyBSD 5.5-DEVELOPMENT daily ISO as of this week benchmarked across DragonFlyBSD 5.4.3 stable, FreeBSD 12.0, Ubuntu 19.04, Red Hat Enterprise Linux 8.0, Debian 9.9, Debian Buster, and CentOS 7 1810 as a wide variety of reference points both from newer and older Linux distributions. (As for no Clear Linux reference point for a speedy reference point, it currently has a regression with AMD + Samsung NVMe SSD support on some hardware, including this box, prohibiting the drive from coming up due to a presumed power management issue that is still being resolved.)
With Matthew Dillon doing much of his development on an AMD Ryzen Threadripper system after he last year proclaimed the greatness of these AMD HEDT CPUs, for this round of testing I also used a Ryzen Threadripper 2990WX with 32 cores / 64 threads. Tests of other AMD/Intel hardware with DragonFlyBSD will come as the next stable release is near and all of the kernel work has settled down. For now it's mostly entertaining our own curiosity how well these DragonFlyBSD optimizations are paying off and how it's increasing the competition against FreeBSD 12 and Linux distributions.

What are the differences between OpenBSD and Linux?

Maybe you have been reading recently about the release of OpenBSD 6.5 and wonder, "What are the differences between Linux and OpenBSD?"
I've also been there at some point in the past and these are my conclusions.
They also apply, to some extent, to other BSDs. However, an important disclaimer applies to this article.
This list is aimed at people who are used to Linux and are curious about OpenBSD. It is written to highlight the most important changes from their perspective, not the absolute most important changes from a technical standpoint.
Please bear with me.

  • A terminal is a terminal is a terminal
  • Practical differences
  • Security and system administration
  • Why philosophical differences matter
  • So what do I choose?
  • How to try OpenBSD ***

News Roundup

NetBSD 2019 Google Summer of Code

We are very happy to announce The NetBSD Foundation Google Summer of Code 2019 projects:

  • Akul Abhilash Pillai - Adapting TriforceAFL for NetBSD kernel fuzzing
  • Manikishan Ghantasala - Add KNF (NetBSD style) clang-format configuration
  • Siddharth Muralee - Enhancing Syzkaller support for NetBSD
  • Surya P - Implementation of COMPAT_LINUX and COMPAT_NETBSD32 DRM ioctls support for NetBSD kernel
  • Jason High - Incorporation of Argon2 Password Hashing Algorithm into NetBSD
  • Saurav Prakash - Porting NetBSD to HummingBoard Pulse
  • Naveen Narayanan - Porting WINE to amd64 architecture on NetBSD

The communiting bonding period - where students get in touch with mentors and community - started yesterday. The coding period will start from May 27 until August 19.
Please welcome all our students and a big good luck to students and mentors! A big thank to Google and The NetBSD Foundation organization mentors and administrators! Looking forward to a great Google Summer of Code!

Reducing that contention

The opening keynote at EuroBSDCon 2016 predicted the future 10 years of BSDs. Amongst all the funny previsions, gnn@FreeBSD said that by 2026 OpenBSD will have its first implementation of SMP. Almost 3 years after this talk, that sounds like a plausible forecast... Why? Where are we? What can we do? Let's dive into the issue!

  • State of affairs

Most of OpenBSD's kernel still runs under a single lock, ze KERNEL_LOCK(). That includes most of the syscalls, most of the interrupt handlers and most of the fault handlers. Most of them, not all of them. Meaning we have collected & fixed bugs while setting up infrastructures and examples. Now this lock remains the principal responsible for the spin % you can observe in top(1) and systat(1).
I believe that we opted for a difficult hike when we decided to start removing this lock from the bottom. As a result many SCSI & Network interrupt handlers as well as all Audio & USB ones can be executed without big lock. On the other hand very few syscalls are already or almost ready to be unlocked, as we incorrectly say. This explains why basic primitives like tsleep(9), csignal() and selwakeup() are only receiving attention now that the top of the Network Stack is running (mostly) without big lock.

  • Next steps

In the past years, most of our efforts have been invested into the Network Stack. As I already mentioned it should be ready to be parallelized. However think we should now concentrate on removing the KERNEL_LOCK(), even if the code paths aren't performance critical.

  • See the Article for the rest of the post

fnaify 1.3 released - more games are "fnaify & run" now

This release finally addresses some of the problems that prevent simple running of several games.
This happens for example when an old FNA.dll library comes with the games that doesn't match the API of our native libraries like SDL2, OpenAL, or MojoShader anymore. Some of those cases can be fixed by simply dropping in a newer FNA.dll. fnaify now asks if FNA 17.12 should be automatically added if a known incompatible FNA version is found. You simply answer yes or no.

Another blocker happens when the game expects to check the SteamAPI - either from a running Steam process, or a bundled steam_api library. OpenBSD 6.5-current now has steamworks-nosteam in ports, a stub library for Steamworks.NET that prevents games from crashing simply because an API function isn't found. The repo is here. fnaify now finds this library in /usr/local/share/steamstubs and uses it instead of the bundled (full) Steamworks.NET.dll.
This may help with any games that use this layer to interact with the SteamAPI, mostly those that can only be obtained via Steam.

vmctl(8): command line syntax changed

The order of the arguments in the create, start, and stop commands of vmctl(8) has been changed to match a commonly expected style. Manual usage or scripting with vmctl must be adjusted to use the new syntax.
For example, the old syntax looked like this:

# vmctl create disk.qcow2 -s 50G

The new syntax specifies the command options before the argument:

# vmctl create -s 50G disk.qcow2

Something that Linux distributions should not do when packaging things

Right now I am a bit unhappy at Fedora for a specific packaging situation, so let me tell you a little story of what I, as a system administrator, would really like distributions to not do.
For reasons beyond the scope of this blog entry, I run a Prometheus and Grafana setup on both my home and office Fedora Linux machines (among other things, it gives me a place to test out various things involving them). When I set this up, I used the official upstream versions of both, because I needed to match what we are running (or would soon be).
Recently, Fedora decided to package Grafana themselves (as a RPM), and they called this RPM package 'grafana'. Since the two different packages are different versions of the same thing as far as package management tools are concerned, Fedora basically took over the 'grafana' package name from Grafana. This caused my systems to offer to upgrade me from the Grafana.com 'grafana-6.1.5-1' package to the Fedora 'grafana-6.1.6-1.fc29' one, which I actually did after taking reasonable steps to make sure that the Fedora version of 6.1.6 was compatible with the file layouts and so on from the Grafana version of 6.1.5.
Why is this a problem? It's simple. If you're going to take over a package name from the upstream, you should keep up with the upstream releases. If you take over a package name and don't keep up to date or keep up to date only sporadically, you cause all sorts of heartburn for system administrators who use the package. The least annoying future of this situation is that Fedora has abandoned Grafana at 6.1.6 and I am going to 'upgrade' it with the upstream 6.2.1, which will hopefully be a transparent replacement and not blow up in my face. The most annoying future is that Fedora and Grafana keep ping-ponging versions back and forth, which will make 'dnf upgrade' into a minefield (because it will frequently try to give me a 'grafana' upgrade that I don't want and that would be dangerous to accept). And of course this situation turns Fedora version upgrades into their own minefield, since now I risk an upgrade to Fedora 30 actually reverting the 'grafana' package version on me.

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv ***
Your browser does not support the HTML5 video tag.

Jun 13 2019

1hr 9mins

Play

Rank #17: 307: Twitching with OpenBSD

Podcast cover
Read more

FreeBSD 11.3 has been released, OpenBSD workstation, write your own fuzzer for the NetBSD kernel, Exploiting FreeBSD-SA-19:02.fd, streaming to twitch using OpenBSD, 3 different ways of dumping hex contents of a file, and more.

Headlines

FreeBSD 11.3-RELEASE Announcement

The FreeBSD Release Engineering Team is pleased to announce the availability of FreeBSD 11.3-RELEASE. This is the fourth release of the stable/11 branch.

  • Some of the highlights:

    • The clang, llvm, lld, lldb, and compiler-rt utilities as well as libc++ have been updated to upstream version 8.0.0.
    • The ELF Tool Chain has been updated to version r3614.
    • OpenSSL has been updated to version 1.0.2s.
    • The ZFS filesystem has been updated to implement parallel mounting.
    • The loader(8) has been updated to extend geli(8) support to all architectures.
    • The pkg(8) utility has been updated to version 1.10.5.
    • The KDE desktop environment has been updated to version 5.15.3.
    • The GNOME desktop environment has been updated to version 3.28.
    • The kernel will now log the jail(8) ID when logging a process exit.
    • Several feature additions and updates to userland applications.
    • Several network driver firmware updates.
    • Warnings for features deprecated in future releases will now be printed on all FreeBSD versions.
    • Warnings have been added for IPSec algorithms deprecated in RFC 8221.
    • Deprecation warnings have been added for weaker algorithms when creating geli(8) providers.
    • And more...

OpenBSD Is Now My Workstation

Why OpenBSD? Simply because it is the best tool for the job for me for my new-to-me Lenovo Thinkpad T420. Additionally, I do care about security and non-bloat in my personal operating systems (business needs can have different priorities, to be clear).

I will try to detail what my reasons are for going with OpenBSD (instead of GNU/Linux, NetBSD, or FreeBSD of which I’m comfortable using without issue), challenges and frustrations I’ve encountered, and what my opinions are along the way.

Disclaimer: in this post, I’m speaking about what is my opinion, and I’m not trying to convince you to use OpenBSD or anything else. I don’t truly care, but wanted to share in case it could be useful to you. I do hope you give OpenBSD a shot as your workstation, especially if it has been a while.

  • A Bit About Me and OpenBSD

I’m not new to OpenBSD, to be clear. I’ve been using it off and on for over 20 years. The biggest time in my life was the early 2000s (I was even the Python port maintainer for a bit), where I not only used it for my workstation, but also for production servers and network devices.

I just haven’t used it as a workstation (outside of a virtual machine) in over 10 years, but have used it for servers. Workstation needs, especially for a primary workstation, are greatly different and the small things end up mattering most.

News Roundup

Write your own fuzzer for NetBSD kernel! [Part 1]

  • How Fuzzing works? The dummy Fuzzer.

The easy way to describe fuzzing is to compare it to the process of unit testing a program, but with different input. This input can be random, or it can be generated in some way that makes it unexpected form standard execution perspective.

The simplest 'fuzzer' can be written in few lines of bash, by getting N bytes from /dev/rand, and putting them to the program as a parameter.

  • Coverage and Fuzzing

What can be done to make fuzzing more effective? If we think about fuzzing as a process, where we place data into the input of the program (which is a black box), and we can only interact via input, not much more can be done.

However, programs usually process different inputs at different speeds, which can give us some insight into the program's behavior. During fuzzing, we are trying to crash the program, thus we need additional probes to observe the program's behaviour.

Additional knowledge about program state can be exploited as a feedback loop for generating new input vectors. Knowledge about the program itself and the structure of input data can also be considered. As an example, if the input data is in the form of HTML, changing characters inside the body will probably cause less problems for the parser than experimenting with headers and HTML tags.

For open source programs, we can read the source code to know what input takes which execution path. Nonetheless, this might be very time consuming, and it would be much more helpful if this can be automated. As it turns out, this process can be improved by tracing coverage of the execution

vBSDcon - CFP - Call for Papers ends July 19th

You can submit your proposal at https://easychair.org/conferences/?conf=vbsdcon2019

The talks will have a very strong technical content bias. Proposals of a business development or marketing nature are not appropriate for this venue.

If you are doing something interesting with a BSD operating system, please submit a proposal. Whether you are developing a very complex system using BSD as the foundation, or helping others and have a story to tell about how BSD played a role, we want to hear about your experience. People using BSD as a platform for research are also encouraged to submit a proposal.

Possible topics include: How we manage a giant installation with respect to handling spam, snd/or sysadmin, and/or networking, Cool new stuff in BSD, Tell us about your project which runs on BSD.

Both users and developers are encouraged to share their experiences.

Exploiting FreeBSD-SA-19:02.fd

In February 2019 the FreeBSD project issued an advisory about a possible vulnerability in the handling of file descriptors. UNIX-like systems such as FreeBSD allow to send file descriptors to other processes via UNIX-domain sockets. This can for example be used to pass file access privileges to the receiving process.

Inside the kernel, file descriptors are used to indirectly reference a C struct which stores the relevant information about the file object. This could for instance include a reference to a vnode which describes the file for the file system, the file type, or the access privileges.

What really happens if a UNIX-domain socket is used to send a file descriptor to another process is that for the receiving process, inside the kernel a reference to this struct is created. As the new file descriptor is a reference to the same file object, all information is inherited. For instance, this can allow to give another process write access to a file on the drive even if the process owner is normally not able to open the file writable.

The advisory describes that FreeBSD 12.0 introduced a bug in this mechanism. As the file descriptor information is sent via a socket, the sender and the receiver have to allocate buffers for the procedure. If the receiving buffer is not large enough, the FreeBSD kernel attempts to close the received file descriptors to prevent a leak of these to the sender. However, while the responsible function closes the file descriptor, it fails to release the reference from the file descriptor to the file object. This could cause the reference counter to wrap.

The advisory further states that the impact of this bug is possibly a local privilege escalation to gain root privileges or a jail escape. However, no proof-of-concept was provided by the advisory authors.

  • In the next section, the bug itself is analyzed to make a statement about the bug class and a guess about a possible exploitation primitive.
  • After that, the bug trigger is addressed.
  • It follows a discussion of three imaginable exploitation strategies - including a discussion of why two of these approaches failed.
  • In the section before last, the working exploit primitive is discussed. It introduces a (at least to the author’s knowledge) new exploitation technique for these kind of vulnerabilities in FreeBSD. The stabilization of the exploit is addressed, too.
  • The last section wraps everything up in a conclusion and points out further steps and challenges.

The privilege escalation is now a piece of cake thanks to a technique used by kingcope, who published a FreeBSD root exploit in 2005, which writes to the file /etc/libmap.conf. This configuration file can be used to hook the loading of dynamic libraries if a program is started. The exploit therefore creates a dynamic library, which copies /bin/sh to another file and sets the suid-bit for the copy. The hooked library is libutil, which is for instance called by su. Therefore, a call to su by the user will afterwards result in a suid copy of /bin/sh.

Streaming to Twitch using OpenBSD

  • Introduction

If you ever wanted to make a twitch stream from your OpenBSD system, this is now possible, thanks to OpenBSD developer thfr@ who made a wrapper named fauxstream using ffmpeg with relevant parameters.

The setup is quite easy, it only requires a few steps and searching on Twitch website two informations, hopefully, to ease the process, I found the links for you.

You will need to make an account on twitch, get your api key (a long string of characters) which should stay secret because it allow anyone having it to stream on your account.

  • These same techniques should work for Twitch, YouTube Live, Periscope, Facebook, etc, including the live streaming service ScaleEngine provides free to BSD user groups.
  • There is also an open source application called ‘OBS’ or Open Broadcaster Studio. It is in FreeBSD ports and should work on all of the other BSDs as well. It has a GUI and supports compositing and green screening. We use it heavily at ScaleEngine and it is also used at JupiterBroadcasting in place of WireCast, a $1000-per-copy commercial application.

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

Jul 18 2019

50mins

Play

Rank #18: 295: Fun with funlinkat()

Podcast cover
Read more

Introducing funlinkat(), an OpenBSD Router with AT&T U-Verse, using NetBSD on a raspberry pi, ZFS encryption is still under development, Rump kernel servers and clients tutorial, Snort on OpenBSD 6.4, and more.

Headlines

Introducing funlinkat

  • It turns out, every file you have ever deleted on a unix machine was probably susceptible to a race condition

One of the first syscalls which was created in Unix-like systems is unlink. In FreeBSD this syscall is number 10 (source) and in Linux, the number is dependent on the architecture but for most of them is also the tenth syscall (source). This indicated that this is one of the primary syscalls. The unlink syscall is very simple and we provide one single path to the file that we want to remove.
The “removing file” process itself is very interesting so let’s spend a moment to understand the it. First, by removing the file we are removing a link from the directory to it. In Unix-like systems we can have many links to a single file (hard links). When we remove all links to the file, the file system will mark the blocks used by the file as free (a different file system will behave differently but let’s not jump into a second digression). This is why the process is called unlinking and not “removing file”. While we unlink the file two or three things will happen:

  • We will remove an entry in the directory with the filename.
  • We will decrease a file reference count (in inode).
  • If links go to zero - the file will be removed from the disk (again this doesn't mean that the blocks from the disk will be filled with zeros, though this may happen depending on the file system and configuration. However, in most cases this means that the file system will mark those blocks to as free and use them to write new data later
    This mostly means that “removing file” from a directory is an operation on the directory and not on the file (inode) itself.
    Another interesting subject is what happens if our system will perform only first or second step from the list. This depends on the file system and this is also something we will leave for another time.
    The problem with the unlink and even unlinkat function is that we don’t have any guarantee of which file we really are unlinking.
    • When you delete a file using its name, you have no guarantee that someone has not already deleted the file, or renamed it, and created a new file with the name you are about to delete.
      We have some stats about the file that we want to unlink. We performed some tests. In the same time another process removed our file and recreated it. When we finally try to remove our file it is no longer the same file. It’s a classic race condition.
    • Many programs will perform checks before trying to remove a file, to make sure it is the correct file, that you have the correct permissions etc. However this exposes the ‘Time-of-Check / Time-of-Use’ class of bugs. I check if the file I am about to remove is the one I created yesterday, it is, so I call unlink() on it. However, between when I checked the date on the file, and when I call unlink, I, some program I am running, might have updated the file. Or a malicious user might have put some other file at that name, so I would be the one who deleted it.
      In Unix-like operating systems we can get a handle for our file called file - a descriptor. File descriptors guarantee us that all the operations that we will be performing on it are done on the same file (inode). Even if someone was to unlink a number of directories entries, the operating system will not free the structures behind the file descriptor, and we can detect the file that was removed by someone and recreated (or just unlinked). So, for example, we have an alternative functions fstat which allows us to get file status of the given descriptor
      We already know that the file may have many links on the disk which point to the single inode. What happens when we open the file? Simplifying: kernel creates a memory representation of the inode (the inode itself is stored on the disk) called vnode. This single representation is used by all processes to refer the inode to the disk. If in a process we open the same file (inode) using different names (for example through hard links) all those files will be linked to the single vnode. That means that the pathname is not stored in the kernel.
      This is basically the reason why we don’t have a funlink function so that instead of the path we are providing just the file descriptor to the file. If we performed the fdunlink syscall, the kernel wouldn’t know which directory entry you would like to remove. Another problem is more architectural: as we discussed earlier unlinking is really an operation on the directory not on the file (inode) itself, so using funlink(fd) may create some confusion because we are not removing the inode corresponding to the file descriptor, we are performing action on the directory which points to the file.
      After some discussion we decided that the only sensible option for FreeBSD would be to create a funlinkat() function. This syscall would only performs additional sanitary checks if we are removing a directory entry which corresponds to the inode stored which refers to the file descriptor.
      int funlinkat(int dfd, const char *path, int fd, int flags);
      The API above will check if the path opened relative to the dfd points to the same vnode. Thanks to that we removed a race condition because all those sanitary checks are performed in the kernel mode while the file system is locked and there is no possibility to change it.
      The fd parameter may be set to the FD_NONE value which will mean that the sanitary check should not be performed and funlinkat will behave just like unlinkat.
      As you can notice I often refer to the unlink syscall but at the end the APIs looks like unlinkat syscall. It is true that the unlink syscall is very old and kind of deprecated. That said I referred to unlink because it’s just simpler. These days unlink simply uses the same code as unlinkat.

Using an OpenBSD Router with AT&T U-Verse

I upgraded to AT&T's U-verse Gigabit internet service in 2017 and it came with an Arris BGW-210 as the WiFi AP and router. The BGW-210 is not a terrible device, but I already had my own Airport Extreme APs wired throughout my house and an OpenBSD router configured with various things, so I had no use for this device. It's also a potentially-insecure device that I can't upgrade or fully disable remote control over.
Fully removing the BGW-210 is not possible as we'll see later, but it is possible to remove it from the routing path. This is how I did it with OpenBSD.

News Roundup

How to use NetBSD on a Raspberry Pi

Do you have an old Raspberry Pi lying around gathering dust, maybe after a recent Pi upgrade? Are you curious about BSD Unix? If you answered "yes" to both of these questions, you'll be pleased to know that the first is the solution to the second, because you can run NetBSD, as far back as the very first release, on a Raspberry Pi.
BSD is the Berkley Software Distribution of Unix. In fact, it's the only open source Unix with direct lineage back to the original source code written by Dennis Ritchie and Ken Thompson at Bell Labs. Other modern versions are either proprietary (such as AIX and Solaris) or clever re-implementations (such as Minix and GNU/Linux). If you're used to Linux, you'll feel mostly right at home with BSD, but there are plenty of new commands and conventions to discover. If you're still relatively new to open source, trying BSD is a good way to experience a traditional Unix.
Admittedly, NetBSD isn't an operating system that's perfectly suited for the Pi. It's a minimal install compared to many Linux distributions designed specifically for the Pi, and not all components of recent Pi models are functional under NetBSD yet. However, it's arguably an ideal OS for the older Pi models, since it's lightweight and lovingly maintained. And if nothing else, it's a lot of fun for any die-hard Unix geek to experience another side of the POSIX world.

ZFS Encryption is still under development (as of March 2019)

One of the big upcoming features that a bunch of people are looking forward to in ZFS is natively encrypted filesystems. This is already in the main development tree of ZFS On Linux, will likely propagate to FreeBSD (since FreeBSD ZFS will be based on ZoL), and will make it to Illumos if the Illumos people want to pull it in. People are looking forward to native encryption so much, in fact, that some of them have started using it in ZFS On Linux already, using either the development tip or one of the 0.8.0 release candidate pre-releases (ZoL is up to 0.8.0-rc3 as of now). People either doing this or planning to do this show up on the ZoL mailing list every so often.

Tutorial On Rump Kernel Servers and Clients

The rump anykernel architecture allows to run highly componentized kernel code configurations in userspace processes. Coupled with the rump sysproxy facility it is possible to run loosely distributed client-server "mini-operating systems". Since there is minimum configuration and the bootstrap time is measured in milliseconds, these environments are very cheap to set up, use, and tear down on-demand.
This document acts as a tutorial on how to configure and use unmodified NetBSD kernel drivers as userspace services with utilities available from the NetBSD base system. As part of this, it presents various use cases. One uses the kernel cryptographic disk driver (cgd) to encrypt a partition. Another one demonstrates how to operate an FFS server for editing the contents of a file system even though your user account does not have privileges to use the host's mount() system call. Additionally, using a userspace TCP/IP server with an unmodified web browser is detailed.

Installing Snort on OpenBSD 6.4

As you may recall from previous posts, I am running an OpenBSD server on an APU2 air-cooled 3 Intel NIC box as my router/firewall for my secure home network. Given that all of my Internet traffic flows through this box, I thought it would be a cool idea to run an Intrusion Detection System (IDS) on it. Snort is the big hog of the open source world so I took a peek in the packages directory on one of the mirrors and lo and behold we have the latest & greatest version of Snort available! Thanks devs!!!
I did some quick Googling and didn’t find much “modern” howto help out there so, after some trial and error, I have it up and running. I thought I’d give back in a small way and share a quickie howto for other Googlers out there who are looking for guidance. Here’s hoping that my title is good enough “SEO” to get you here!

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv


Your browser does not support the HTML5 video tag.

Apr 25 2019

1hr 1min

Play

Rank #19: 297: Dragonfly In The Wild

Podcast cover
Read more

FreeBSD ZFS vs. ZoL performance, Dragonfly 5.4.2 has been release, containing web services with iocell, Solaris 11.4 SRU8, Problem with SSH Agent forwarding, OpenBSD 6.4 to 6.5 upgrade guide, and more.

Headlines

FreeBSD ZFS vs. ZoL Performance, Ubuntu ZFS On Linux Reference

With iX Systems having released new images of FreeBSD reworked with their ZFS On Linux code that is in development to ultimately replace their existing FreeBSD ZFS support derived from the code originally found in the Illumos source tree, here are some fresh benchmarks looking at the FreeBSD 12 performance of ZFS vs. ZoL vs. UFS and compared to Ubuntu Linux on the same system with EXT4 and ZFS.
Using an Intel Xeon E3-1275 v6 with ASUS P10S-M WS motherboard, 2 x 8GB DDR4-2400 ECC UDIMMs, and Samsung 970 EVO Plus 500GB NVMe solid-state drive was used for all of this round of testing. Just a single modern NVMe SSD was used for this round of ZFS testing while as the FreeBSD ZoL code matures I'll test on multiple systems using a more diverse range of storage devices.
FreeBSD 12 ZoL was tested using the iX Systems image and then fresh installs done of FreeBSD 12.0-RELEASE when defaulting to the existing ZFS root file-system support and again when using the aging UFS file-system. Ubuntu 18.04.2 LTS with the Linux 4.18 kernel was used when testing its default EXT4 file-system and then again when using the Ubuntu-ZFS ZoL support. Via the Phoronix Test Suite various BSD/Linux I/O benchmarks were carried out.
Overall, the FreeBSD ZFS On Linux port is looking good so far and we are looking forward to it hopefully maturing in time for FreeBSD 13.0. Nice job to iX Systems and all of those involved, especially the ZFS On Linux project. Those wanting to help in testing can try the FreeBSD ZoL spins. Stay tuned for more benchmarks and on more diverse hardware as time allows and the FreeBSD ZoL support further matures, but so far at least the performance numbers are in good shape.

DragonFlyBSD 5.4.2 is out

Upgrading guide

Here's the tag commit, for what has changed from 5.4.1 to 5.4.2
The normal ISO and IMG files are available for download and install, plus an uncompressed ISO image for those installing remotely. I uploaded them to mirror-master.dragonflybsd.org last night so they should be at your local mirror or will be soon. This version includes Matt's fix for the HAMMER2 corruption bug he identified recently.
If you have an existing 5.4 system and are running a generic kernel, the normal upgrade process will work.

> cd /usr/src
> git pull
> make buildworld.
> make buildkernel.
> make installkernel.
> make installworld
> make upgrade

After your next reboot, you can optionally update your rescue system:

> cd /usr/src
> make initrd

As always, make sure your packages are up to date:

> pkg update
> pkg upgrade

News Roundup

Containing web services with iocell

I'm a huge fan of the FreeBSD jails feature. It is a great system for splitting services into logical units with all the performance of the bare metal system. In fact, this very site runs in its own jail! If this is starting to sound like LXC or Docker, it might surprise you to learn that OS-level virtualization has existed for quite some time. Kudos to the Linux folks for finally getting around to it. 😛
If you're interested in the history behind Jails, there is an excellent talk from Papers We Love on the subject: https://www.youtube.com/watch?v=hgN8pCMLI2U

  • Getting started

There are plenty of options when it comes to setting up the jail system. Ezjail and Iocage seem popular, or you could do things manually. Iocage was recently rewritten in python, but was originally a set of shell scripts. That version has since been forked under the name Iocell, and I think it's pretty neat, so this tutorial will be using Iocell.

  • To start, you'll need the following:
    • A FreeBSD install (we'll be using 11.0)
    • The iocell package (available as a package, also in the ports tree)
    • A ZFS pool for hosting the jails

Once you have installed iocell and configured your ZFS pool, you'll need to run a few commands before creating your first jail. First, tell iocell which ZFS pool to use by issuing iocell activate $POOLNAME. Iocell will create a few datasets.

As you can imagine, your jails are contained within the /iocell/jails dataset. The /iocell/releases dataset is used for storing the next command we need to run, iocell fetch. Iocell will ask you which release you'd like to pull down. Since we're running 11.0 on the host, pick 11.0-RELEASE. Iocell will download the necessary txz files and unpack them in /iocell/releases.

  • See Article for the rest of the walkthrough.

Oracle Solaris 11.4 SRU8

Today we are releasing the SRU 8 for Oracle Solaris 11.4. It is available via 'pkg update' from the support repository or by downloading the SRU from My Oracle Support Doc ID 2433412.1.

  • This SRU introduces the following enhancements:
    • Integration of 28060039 introduced an issue where any firmware update/query commands will log eereports and repeated execution of such commands led to faulty/degraded NIC. The issue has been addressed in this SRU.
    • UCB (libucb, librpcsoc, libdbm, libtermcap, and libcurses) libraries have been reinstated for Oracle Solaris 11.4
    • Re-introduction of the service fc-fabric.
    • ibus has been updated to 1.5.19
  • The following components have also been updated to address security issues:
    • NTP has been updated to 4.2.8p12
    • Firefox has been updated to 60.6.0esr
    • BIND has been updated to 9.11.6
    • OpenSSL has been updated to 1.0.2r
    • MySQL has been updated to 5.6.43 & 5.7.25
    • libxml2 has been updated to 2.9.9
    • libxslt has been updated to 1.1.33
    • Wireshark has been updated to 2.6.7
    • ncurses has been updated to 6.1.0.20190105
    • Apache Web Server has been updated to 2.4.38
    • perl 5.22
    • pkg.depot

The Problem with SSH Agent Forwarding

After hacking the matrix.org website today, the attacker opened a series of GitHub issues mentioning the flaws he discovered. In one of those issues, he mentions that “complete compromise could have been avoided if developers were prohibited from using [SSH agent forwarding].”
Here’s what man ssh_config has to say about ForwardAgent: "Agent forwarding should be enabled with caution. Users with the ability to bypass file permissions on the remote host (for the agent’s Unix-domain socket) can access the local agent through the forwarded connection. An attacker cannot obtain key material from the agent, however they can perform operations on the keys that enable them to authenticate using the identities loaded into the agent.""
Simply put: if your jump box is compromised and you use SSH agent forwarding to connect to another machine through it, then you risk also compromising the target machine!
Instead, you should use either ProxyCommand or ProxyJump (added in OpenSSH 7.3). That way, ssh will forward the TCP connection to the target host via the jump box and the actual connection will be made on your workstation. If someone on the jump box tries to MITM your connection, then you will be warned by ssh.

[OpenBSD Upgrade Guide: 6.4 to 6.5

Start by performing the pre-upgrade steps. Next, boot from the install kernel, bsd.rd: use bootable install media, or place the 6.5 version of bsd.rd in the root of your filesystem and instruct the boot loader to boot this kernel. Once this kernel is booted, choose the (U)pgrade option and follow the prompts. Apply the configuration changes and remove the old files. Finish up by upgrading the packages: pkg_add -u.
Alternatively, you can use the manual upgrade process.
You may wish to check the errata page or upgrade to the stable branch to get any post-release fixes.

  • Before rebooting into the install kernel
  • Configuration and syntax changes
  • Files to remove
  • Special packages
  • Upgrade without the install kernel

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv


Your browser does not support the HTML5 video tag.

May 09 2019

40mins

Play

Rank #20: 327: ZFS Rename Repo

Podcast cover
Read more

We read FreeBSD’s third quarterly status report, OpenBSD on Sparc64, ZoL repo move to OpenZFS, GEOM NOP, keeping NetBSD up-to-date, and more.

Headlines

FreeBSD third quarterly status report for 2019

This quarter the reports team has been more active than usual thanks to a better organization: calls for reports and reminders have been sent regularly, reports have been reviewed and merged quickly (I would like to thank debdrup@ in particular for his reviewing work).

Efficiency could still be improved with the help of our community. In particular, the quarterly team has found that many reports have arrived in the last days before the deadline or even after. I would like to invite the community to follow the guidelines below that can help us sending out the reports sooner.

Starting from next quarter, all quarterly status reports will be prepared the last month of the quarter itself, instead of the first month after the quarter's end. This means that deadlines for submitting reports will be the 1st of January, April, July and October.

Next quarter will then be a short one, covering the months of November and December only and the report will probably be out in mid January.

OpenBSD on Sparc64

OpenBSD, huh? Yes, I usually write about FreeBSD and that’s in fact what I tried installing on the machine first. But I ran into problems with it very early on (never even reached single user mode) and put it aside for later. Since I powered up the SunFire again last month, I needed an OS now and chose OpenBSD for the simple reason that I have it available.

First I wanted to call this article simply “OpenBSD on SPARC” – but that would have been misleading since OpenBSD used to support 32-bit SPARC processors, too. The platform was just put to rest after the 5.9 release.

Version 6.0 was the last release of OpenBSD that came on CD-ROM. When I bought it, I thought that I’d never use the SPARC CD. But here was the chance! While it is an obsolete release, it comes with the cryptographic signatures to verify the next release. So the plan is to start at 6.0 as I can trust the original CDs and then update to the latest release. This will also be an opportunity to recap on some of the things that changed over the various versions.

News Roundup

ZoL repo move to OpenZFS

Because it will contain the ZFS source code for both Linux and FreeBSD, we will rename the "ZFSonLinux" code repository to "OpenZFS". Specifically, the repo at http://github.com/ZFSonLinux/zfs will be moved to the OpenZFS organization, at http://github.com/OpenZFS/zfs.

The next major release of ZFS for Linux and FreeBSD will be "OpenZFS 2.0", and is expected to ship in 2020.

Mcclure111 Sun Thread

A long time ago— like 15 years ago— I worked at Sun Microsystems. The company was nearly dead at the time (it died a couple years later) because they didn't make anything that anyone wanted to buy anymore. So they had a lot of strange ideas about how they'd make their comeback.

GEOM NOP

Sometimes while testing file systems or applications you want to simulate some errors on the disk level. The first time I heard about this need was from Baptiste Daroussin during his presentation at AsiaBSDCon 2016. He mentioned how they had built a test lab with it. The same need was recently discussed during the PGCon 2019, to test a PostgreSQL instance. If you are FreeBSD user, I have great news for you: there is a GEOM provider which allows you to simulate a failing device.

GNOP allows us to configure transparent providers from existing ones. The first interesting option of it is that we can slice the device into smaller pieces, thanks to the ‘offset option’ and ‘stripsesize’. This allows us to observe how the data on the disk is changing. Let’s assume that we want to observe the changes in the GPT table when the GPT flags are added or removed (for example the bootme flags which are described here). We can use dd every time and analyze it using absolute values from the disks.

Keeping NetBSD up-to-date with pkg_comp 2.0

This is a tutorial to guide you through the shiny new pkg_comp 2.0 on NetBSD.

Goals: to use pkg_comp 2.0 to build a binary repository of all the packages you are interested in; to keep the repository fresh on a daily basis; and to use that repository with pkgin to maintain your NetBSD system up-to-date and secure.

This tutorial is specifically targeted at NetBSD but should work on other platforms with some small changes. Expect, at the very least, a macOS-specific tutorial as soon as I create a pkg_comp standalone installer for that platform.

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

Dec 05 2019

1hr 23mins

Play

333: Unix Keyboard Joy

Podcast cover
Read more

Your Impact on FreeBSD in 2019, Wireguard on OpenBSD Router, Amazon now has FreeBSD/ARM 12, pkgsrc-2019Q4, The Joys of UNIX Keyboards, OpenBSD on Digital Ocean, and more.

Headlines

Your Impact on FreeBSD in 2019

It’s hard to believe that 2019 is nearly over. It has been an amazing year for supporting the FreeBSD Project and community! Why do I say that? Because as I reflect over the past 12 months, I realize how many events we’ve attended all over the world, and how many lives we’ve touched in so many ways. From advocating for FreeBSD to implementing FreeBSD features, my team has been there to help make FreeBSD the best open source project and operating system out there.

In 2019, we focused on supporting a few key areas where the Project needed the most help. The first area was software development. Whether it was contracting FreeBSD developers to work on projects like wifi support, to providing internal staff to quickly implement hardware workarounds, we’ve stepped in to help keep FreeBSD innovative, secure, and reliable. Software development includes supporting the tools and infrastructure that make the development process go smoothly, and we’re on it with team members heading up the Continuous Integration efforts, and actively involved in the clusteradmin and security teams.

Our advocacy efforts focused on recruiting new users and contributors to the Project. We attended and participated in 38 conferences and events in 21 countries. From giving FreeBSD presentations and workshops to staffing tables, we were able to have 1:1 conversations with thousands of attendees.

Our travels also provided opportunities to talk directly with FreeBSD commercial and individual users, contributors, and future FreeBSD user/contributors. We’ve seen an increase in use and interest in FreeBSD from all of these organizations and individuals. These meetings give us a chance to learn more about what organizations need and what they and other individuals are working on. The information helps inform the work we should fund.

Wireguard on OpenBSD Router

wireguard (wg) is a modern vpn protocol, using the latest class of encryption algorithms while at the same time promising speed and a small code base.

modern crypto and lean code are also tenants of openbsd, thus it was a no brainer to migrate my router from openvpn over to wireguard.

my setup : a collection of devices, both wired and wireless, that are nat’d through my router (openbsd 6.6) out via my vpn provider azire* and out to the internet using wg-quick to start wg.

running : doubtless this could be improved on, but currently i start wg manually when my router boots. this, and the nat'ing on the vpn interface mean its impossible for clients to connect to the internet without the vpn being up. as my router is on a ups and only reboots when a kernel patch requires it, it’s a compromise i can live with. run wg-quick (please replace vpn with whatever you named your wg .conf file.) and reload pf rules.

News Roundup

Amazon now has FreeBSD/ARM 12

AWS, the cloud division of Amazon, announced in December the next generation of its ARM processors, the Graviton2. This is a custom chip design with a 7nm architecture. It is based on 64-bit ARM Neoverse cores.

Compared to first-generation Graviton processors (A1), today’s new chips should deliver up to 7x the performance of A1 instances in some cases. Floating point performance is now twice as fast. There are additional memory channels and cache speed memory access should be much faster.

The company is working on three types of Graviton2 EC2 instances that should be available soon. Instances with a “g” suffix are powered by Graviton2 chips. If they have a “d” suffix, it also means that they have NVMe local storage.

  • General-purpose instances (M6g and M6gd)

  • Compute-optimized instances (C6g and C6gd)

  • Memory-optimized instances (R6g and R6gd)

You can choose instances with up to 64 vCPUs, 512 GiB of memory and 25 Gbps networking.

And you can see that ARM-powered servers are not just a fad. AWS already promises a 40% better price/performance ratio with ARM-based instances when you compare them with x86-based instances.

AWS has been working with operating system vendors and independent software vendors to help them release software that runs on ARM. ARM-based EC2 instances support Amazon Linux 2, Ubuntu, Red Hat, SUSE, Fedora, Debian and FreeBSD. It also works with multiple container services (Docker, Amazon ECS, and Amazon Elastic Kubernetes Service).

Announcing the pkgsrc-2019Q4 release

The pkgsrc developers are proud to announce the 65th quarterly release of pkgsrc, the cross-platform packaging system. pkgsrc is available with more than 20,000 packages, running on 23 separate platforms; more information on pkgsrc itself is available at https://www.pkgsrc.org/

In total, 190 packages were added, 96 packages were removed, and 1,868 package updates (to 1388 unique packages) were processed since the pkgsrc-2019Q3 release. As usual, a large number of updates and additions were processed for packages for go (14), guile (11), perl (170), php (10), python (426), and ruby (110). This continues pkgsrc's tradition of adding useful packages, updating many packages to more current versions, and pruning unmaintained packages that are believed to have essentially no users.

The Joys of UNIX Keyboards

I fell in love with a dead keyboard layout.

A decade or so ago while helping a friends father clean out an old building, we came across an ancient Sun Microsystems server. We found it curious. Everything about it was different from what we were used to. The command line was black on white, the connectors strange and foreign, and the keyboard layout was bizarre.

We never did much with it; turning it on made all the lights in his home dim, and our joint knowledge of UNIX was nonexistent. It sat in his bedroom for years supporting his television at the foot of his bed.

I never forgot that keyboard though. The thought that there was this alternative layout out there seemed intriguing to me.

OpenBSD on Digital Ocean

Last night I had a need to put together a new OpenBSD machine. Since I already use DigitalOcean for one of my public DNS servers I wanted to use them for this need but sadly like all too many of the cloud providers they don't support OpenBSD. Now they do support FreeBSD and I found a couple writeups that show how to use FreeBSD as a shim to install OpenBSD.

They are both sort of old at this point and with OpenBSD 6.6 out I ran into a bit of a snag. The default these days is to use a GPT partition table to enable EFI booting. This is generally pretty sane but it looks to me like the FreeBSD droplet doesn't support this. After the installer rebooted the VM failed to boot, being unable to find the bootloader.

Thankfully DigitalOcean has a recovery ISO that you can boot by simply switching to it and powering off and then on your Droplet.

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

Jan 16 2020

40mins

Play

332: The BSD Hyperbole

Podcast cover
Read more

Announcing HyperbolaBSD, IPFW In-Kernel NAT setup on FreeBSD, Wayland and WebRTC enabled for NetBSD 9/Linux, LLDB Threading support ready for mainline, OpenSSH U2F/FIDO support in base, Dragonfly drm/i915: Update, and more.

Headlines

HyperbolaBSD Announcement

Due to the Linux kernel rapidly proceeding down an unstable path, we are planning on implementing a completely new OS derived from several BSD implementations.

This was not an easy decision to make, but we wish to use our time and resources to create a viable alternative to the current operating system trends which are actively seeking to undermine user choice and freedom.

This will not be a "distro", but a hard fork of the OpenBSD kernel and userspace including new code written under GPLv3 and LGPLv3 to replace GPL-incompatible parts and non-free ones.

  • Reasons for this include:
    • Linux kernel forcing adaption of DRM, including HDCP.
    • Linux kernel proposed usage of Rust (which contains freedom flaws and a centralized code repository that is more prone to cyber attack and generally requires internet access to use.)
    • Linux kernel being written without security and in mind. (KSPP is basically a dead project and Grsec is no longer free software)
    • Many GNU userspace and core utils are all forcing adaption of features without build time options to disable them. E.g. (PulseAudio / SystemD / Rust / Java as forced dependencies)
    • As such, we will continue to support the Milky Way branch until 2022 when our legacy Linux-libre kernel reaches End of Life.

Future versions of Hyperbola will be using HyperbolaBSD which will have the new kernel, userspace and not be ABI compatible with previous versions.

HyperbolaBSD is intended to be modular and minimalist so other projects will be able to re-use the code under free license.

A simple IPFW In-Kernel NAT setup on FreeBSD

After graduating college, I am moving from Brooklyn, NY to Redmond, WA (guess where I got a job). I always wanted to re-do my OPNsense firewall (currently a HP T730) with stock FreeBSD and IPFW’s in-kernel NAT.

Why IPFW? Benchmarks have shown IPFW to be faster which is especially good for my Tor relay, and because I can! However, one downside of IPFW is less documentation vs PF, even less without natd (which we’re not using), and this took me time to figure this out.

But since my T730 is already packed, I am testing this on a old PC with two NICs, and my laptop [1] as a client with an USB-to-Ethernet adapter.

News Roundup

HEADS UP: Wayland and WebRTC enabled for NetBSD 9/Linux

This is just a heads up that the Wayland option is now turned on by

default for NetBSD 9 and Linux in cases where it peacefully coexists
with X11.

  • Right now, this effects the following packages:
    • graphics/MesaLib
    • devel/SDL2
    • www/webkit-gtk
    • x11/gtk3

The WebRTC option has also been enabled by default on NetBSD 9 for two Firefox versions: www/firefox, www/firefox68

Please keep me informed of any fallout. Hopefully, there will be none.

If you want to try out Wayland-related things on NetBSD 9, wm/velox/MESSAGE may be interesting for you.

LLDB Threading support now ready for mainline

Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages.

In February, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support, extending NetBSD's ptrace interface to cover more register types and fix compat32 issues and fixing watchpoint support. Then, I've started working on improving thread support which is taking longer than expected. You can read more about that in my September 2019 report.

So far the number of issues uncovered while enabling proper threading support has stopped me from merging the work-in-progress patches. However, I've finally reached the point where I believe that the current work can be merged and the remaining problems can be resolved afterwards. More on that and other LLVM-related events happening during the last month in this report.

OpenSSH U2F/FIDO support in base

Hardware backed keys can be generated using "ssh-keygen -t ecdsa-sk" (or "ed25519-sk" if your token supports it). Many tokens require to be touched/tapped to confirm this step.

You'll get a public/private keypair back as usual, except in this case, the private key file does not contain a highly-sensitive private key but instead holds a "key handle" that is used by the security key to derive the real private key at signing time.

So, stealing a copy of the private key file without also stealing your security key (or access to it) should not give the attacker anything.

drm/i915: Update to Linux 4.8.17

  • drm/i915: Update to Linux 4.8.17
    • Broxton, Valleyview and Cherryview support improvements
    • Broadwell and Gen9/Skylake support improvements
    • Broadwell brightness fixes from OpenBSD
    • Atomic modesetting improvements
    • Various bug fixes and performance enhancements

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Your browser does not support the HTML5 video tag.

Jan 09 2020

45mins

Play

331: Why Computers Suck

Podcast cover
Read more

How learning OpenBSD makes computers suck a little less, How Unix works, FreeBSD 12.1 Runs Well on Ryzen Threadripper 3970X, BSDCan CFP, HardenedBSD Infrastructure Goals, and more.

Headlines

Why computers suck and how learning from OpenBSD can make them marginally less horrible

How much better could things actually be if we abandoned the enterprise development model?

Next I will compare this enterprise development approach with non-enterprise development - projects such as OpenBSD, which do not hesitate to introduce ABI breaking changes to improve the codebase.

One of the most commonly referred to pillars of the project's philosophy has long been its emphasis on clean functional code. Any code which makes it into OpenBSD is subject to ongoing aggressive audits for deprecated, or otherwise unmaintained code in order to reduce cruft and attack surface. Additionally the project creator, Theo de Raadt, and his team of core developers engage in ongoing development for proactive mitigations for various attack classes many of which are directly adopted by various multi-platform userland applications as well as the operating systems themselves (Windows, Linux, and the other BSDs). Frequently it is the case that introducing new features (not just deprecating old ones) introduces new incompatibilities against previously functional binaries compiled for OpenBSD.

To prevent the sort of kernel memory bloat that has plagued so many other operating systems for years, the project enforces a hard ceiling on the number of lines of code that can ever be in ring 0 at a given time. Current estimates guess the number of bugs per line of code in the Linux kernel are around 1 bug per every 10,000 lines of code. Think of this in the context of the scope creep seen in the Linux kernel (which if I recall correctly is currently at around 100,000,000 lines of code), as well as the Windows NT kernel (500,000,000 lines of code) and you quickly begin to understand how adding more and more functionality into the most privileged components of the operating system without first removing old components begins to add up in terms of the drastic difference seen between these systems in the number of zero day exploits caught in the wild respectively.

How Unix Works: Become a Better Software Engineer

Unix is beautiful. Allow me to paint some happy little trees for you. I’m not going to explain a bunch of commands – that’s boring, and there’s a million tutorials on the web doing that already. I’m going to leave you with the ability to reason about the system.

Every fancy thing you want done is one google search away.

But understanding why the solution does what you want is not the same.

That’s what gives you real power, the power to not be afraid.

And since it rhymes, it must be true.

News Roundup

FreeBSD 12.1 Runs Refreshingly Well With AMD Ryzen Threadripper 3970X

For those of you interested in AMD's new Ryzen Threadripper 3960X/3970X processors with TRX40 motherboards for running FreeBSD, the experience in our initial testing has been surprisingly pleasant. In fact, it works out-of-the-box which one could argue is better than the current Linux support that needs the MCE workaround for booting. Here are some benchmarks of FreeBSD 12.1 on the Threadripper 3970X compared to Linux and Windows for this new HEDT platform.

It was refreshing to see FreeBSD 12.1 booting and running just fine with the Ryzen Threadripper 3970X 32-core/64-thread processor from the ASUS ROG ZENITH II EXTREME motherboard and all core functionality working including the PCIe 4.0 NVMe SSD storage, onboard networking, etc. The system was running with 4 x 16GB DDR4-3600 memory, 1TB Corsair Force MP600 NVMe SSD, and Radeon RX 580 graphics. It was refreshing to see FreeBSD 12.1 running well with this high-end AMD Threadripper system considering Linux even needed a boot workaround.

While the FreeBSD 12.1 experience was trouble-free with the ASUS TRX40 motherboard (ROG Zenith II Extreme) and AMD Ryzen Threadripper 3970X, DragonFlyBSD unfortunately was not. Both DragonFlyBSD 5.6.2 stable and the DragonFlyBSD daily development snapshot from last week were yielding a panic on boot. So with that, DragonFlyBSD wasn't tested for this Threadripper 3970X comparison but just FreeBSD 12.1.

FreeBSD 12.1 on the Threadripper 3970X was benchmarked both with its default LLVM Clang 8.0.1 compiler and again with GCC 9.2 from ports for ruling out compiler differences. The FreeBSD 12.1 performance was compared to last week's Windows 10 vs. Linux benchmarks with the same system.

BSDCan 2020 CFP

BSDCan 2020 will be held 5-6 (Fri-Sat) June, 2020 in Ottawa, at the University of Ottawa. It will be preceded by two days of tutorials on 3-4 June (Wed-Thu).

NOTE the change of month in 2020 back to June Also: do not miss out on the Goat BOF on Tuesday 2 June.

We are now accepting proposals for talks. The talks should be designed with a very strong technical content bias. Proposals of a business development or marketing nature are not appropriate for this venue.

If you are doing something interesting with a BSD operating system, please submit a proposal. Whether you are developing a very complex system using BSD as the foundation, or helping others and have a story to tell about how BSD played a role, we want to hear about your experience. People using BSD as a platform for research are also encouraged to submit a proposal. Possible topics include:

  • How we manage a giant installation with respect to handling spam.
  • and/or sysadmin.
  • and/or networking.
  • Cool new stuff in BSD
  • Tell us about your project which runs on BSD
  • other topics (see next paragraph)

From the BSDCan website, the Archives section will allow you to review the wide variety of past BSDCan presentations as further examples.

Both users and developers are encouraged to share their experiences.

HardenedBSD Infrastructure Goals

2019 has been an extremely productive year with regards to HardenedBSD's infrastructure. Several opportunities aligned themselves in such a way as to open a door for a near-complete rebuild with a vast expansion.

The last few months especially have seen a major expansion of our infrastructure. We obtained a number of to-be-retired Dell R410 servers. The crash of our nightly build server provided the opportunity to deploy these R410 servers, doubling our build capacity.

My available time to spend on HardenedBSD has decreased compared to this time last year. As part of rebuilding our infrastructure, I wanted to enable the community to be able to contribute. I'm structuring the work such that help is just a pull request away. Those in the HardenedBSD community who want to contribute to the infrastructure work can simply open a pull request. I'll review the code, and deploy it after a successful review. Users/contributors don't need access to our servers in order to improve them.

My primary goal for the rest of 2019 and into 2020 is to become fully self-hosted, with the sole exception of email. I want to transition the source-of-truth git repos to our own infrastructure. We will still provide a read-only mirror on GitHub.

As I develop this infrastructure, I'm doing so with human rights in mind. HardenedBSD is in a very unique position. In 2020, I plan to provide production Tor Onion Services for the various bits of our infrastructure. HardenedBSD will provide access to its various internal services to its developers and contributors. The entire development lifecycle, going from dev to prod, will be able to happen over Tor.

Transparency will be key moving forward. Logs for the auto-sync script are now published directly to GitHub. Build logs will be, soon, too. Logs of all automated processes, and the code for those processes, will be tracked publicly via git. This will be especially crucial for development over Tor.

Integrating Tor into our infrastructure so deeply increases risk and maintenance burden. However, I believe that through added transparency, we will be able to mitigate risk. Periodic audits will need to be performed and published.

I hope to migrate HardenedBSD's site away from Drupal to a static site generator. We don't really need the dynamic capabilities Drupal gives us. The many security issues Drupal and PHP both bring also leave much to be desired.

So, that's about it. I spent the last few months of 2019 laying the foundation for a successful 2020. I'm excited to see how the project grows.

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

Jan 02 2020

1hr 9mins

Play

330: Happy Holidays, All(an)

Podcast cover
Read more

Authentication Vulnerabilities in OpenBSD, NetBSD 9.0 RC1 is available, Running FreeNAS on a DigitalOcean droplet, NomadBSD 1.3 is here, at e2k19 nobody can hear you scream, and more.

Headlines

Authentication vulnerabilities in OpenBSD

  • We discovered an authentication-bypass vulnerability in OpenBSD's authentication system: this vulnerability is remotely exploitable in smtpd, ldapd, and radiusd, but its real-world impact should be studied on a case-by-case basis. For example, sshd is not exploitable thanks to its defense-in-depth mechanisms.
  • From the manual page of login.conf:

OpenBSD uses BSD Authentication, which is made up of a variety of authentication styles. The authentication styles currently provided are:
passwd Request a password and check it against the password in the master.passwd file. See login_passwd(8).
skey Send a challenge and request a response, checking it with S/Key (tm) authentication. See login_skey(8).
yubikey Authenticate using a Yubico YubiKey token. See login_yubikey(8).
For any given style, the program /usr/libexec/auth/login_style is used to
perform the authentication. The synopsis of this program is:
/usr/libexec/auth/login_style [-v name=value] [-s service] username class

  • This is the first piece of the puzzle: if an attacker specifies a username of the form "-option", they can influence the behavior of the authentication program in unexpected ways.
login_passwd [-s service] [-v wheel=yes|no] [-v lastchance=yes|no] user [class] The service argument specifies which protocol to use with the invoking program. The allowed protocols are login, challenge, and response. (The challenge protocol is silently ignored but will report success as passwd-style authentication is not challenge-response based).
  • This is the second piece of the puzzle: if an attacker specifies the username "-schallenge" (or "-schallenge:passwd" to force a passwd-style authentication), then the authentication is automatically successful and therefore bypassed.
  • Case study: smtpd
  • Case study: ldapd
  • Case study: radiusd
  • Case study: sshd
  • Acknowledgments: We thank Theo de Raadt and the OpenBSD developers for their incredibly quick response: they published patches for these vulnerabilities less than 40 hours after our initial contact. We also thank MITRE's CVE Assignment Team.

First release candidate for NetBSD 9.0 available!

  • Since the start of the release process four months ago a lot of improvements went into the branch - more than 500 pullups were processed!
  • This includes usbnet (a common framework for usb ethernet drivers), aarch64 stability enhancements and lots of new hardware support, installer/sysinst fixes and changes to the NVMM (hardware virtualization) interface.
  • We hope this will lead to the best NetBSD release ever (only to be topped by NetBSD 10 next year).
  • Here are a few highlights of the new release:

    Support for Arm AArch64 (64-bit Armv8-A) machines, including "Arm ServerReady"
    compliant machines (SBBR+SBSA)
    Enhanced hardware support for Armv7-A
    Updated GPU drivers (e.g. support for Intel Kabylake)
    Enhanced virtualization support
    Support for hardware-accelerated virtualization (NVMM)
    Support for Performance Monitoring Counters
    Support for Kernel ASLR
    Support several kernel sanitizers (KLEAK, KASAN, KUBSAN)
    Support for userland sanitizers
    Audit of the network stack
    Many improvements in NPF
    Updated ZFS
    Reworked error handling and NCQ support in the SATA subsystem
    Support a common framework for USB Ethernet drivers (usbnet)

  • More information on the RC can be found on the NetBSD 9 release page

News Roundup

Running FreeNAS on a Digitalocean droplet

  • ZFS is awesome. FreeBSD even more so. FreeNAS is the battle-tested, enterprise-ready-yet-home-user-friendly software defined storage solution which is cooler then deep space, based on FreeBSD and makes heavy use of ZFS. This is what I (and soooooo many others) use for just about any storage-related task. I can go on and on and on about what makes it great, but if you're here, reading this, you probably know all that already and we can skip ahead.
  • I've needed an offsite FreeNAS setup to replicate things to, to run some things, to do some stuff, basically, my privately-owned, tightly-controlled NAS appliance in the cloud, one I control from top to bottom and with support for whatever crazy thing I'm trying to do. Since I'm using DigitalOcean as my main VPS provider, it seemed logical to run FreeNAS there, however, you can't. While DO supports many many distos and pre-setup applications (e.g OpenVPN), FreeNAS isn't a supported feature, at least not in the traditional way :)
  • Before we begin, here's the gist of what we're going to do:

Base of a FreeBSD droplet, we'll re-image our boot block device with FreeNAS iso. We'll then install FreeNAS on the second block device. Once done we're going to do the ol' switcheroo: we're going to re-image our original boot block device using the now FreeNAS-installed second block device.

  • Part 1: re-image our boot block device to boot FreeNAS install media.
  • Part 2: Install FreeNAS on the second block-device
  • Part 3: Re-image the boot block device using the FreeNAS-installed block device

NomadBSD 1.3 is now available

  • From the release notes:

The base system has been changed to FreeBSD 12.1-RELEASE-p1
Due to a deadlock problem, FreeBSD's unionfs has been replaced by unionfs-fuse
The GPT layout has been changed to MBR. This prevents problems with Lenovo
systems that refuse to boot from GPT if "lenovofix" is not set, and systems that
hang on boot if "lenovofix" is set.
Support for ZFS installations has been added to the NomadBSD installer.
The rc-script for setting up the network interfaces has been fixed and improved.
Support for setting the country code for the wlan device has been added.
Auto configuration for running in VirtualBox has been added.
A check for the default display has been added to the graphics configuration scripts. This fixes problems where users with Optimus have their NVIDIA card disabled, and use the integrated graphics chip instead.
NVIDIA driver version 440 has been added.
nomadbsd-dmconfig, a Qt tool for selecting the display manager theme, setting the
default user and autologin has been added.
nomadbsd-adduser, a Qt tool for added preconfigured user accounts to the system has been added.
Martin Orszulik added Czech translations to the setup and installation wizard.
The NomadBSD logo, designed by Ian Grindley, has been changed.
Support for localized error messages has been added.
Support for localizing the password prompts has been added.
Some templates for starting other DEs have been added to ~/.xinitrc.
The interfaces of nomadbsd-setup-gui and nomadbsd-install-gui have been improved.
A script that helps users to configure a multihead systems has been added.
The Xorg driver for newer Intel GPUs has been changed from "intel" to "modesetting".
/proc has been added to /etc/fstab
A D-Bus session issue has been fixed which prevented thunar from accessing samba shares.
DSBBg which allows users to change and manage wallpapers has been added.
The latest version of update_obmenu now supports auto-updating the Openbox menu. Manually updating the Openbox menu after packet (de)installation is therefore no longer needed.

Support for multiple keyboard layouts has been added.
www/palemoon has been removed.
mail/thunderbird has been removed.
audio/audacity has been added.
deskutils/orage has been added.
the password manager fpm2 has been replaced by KeePassXC
mail/sylpheed has been replaced by mail/claws-mail
multimedia/simplescreenrecorder has been added.
DSBMC has been changed to DSBMC-Qt
Many small improvements and bug fixes.

At e2k19 nobody can hear you scream

  • After 2 years it was once again time to pack skis and snowshoes, put a satellite dish onto a sledge and hike through the snowy rockies to the Elk Lakes hut.
  • I did not really have much of a plan what I wanted to work on but there were a few things I wanted to look into. One of them was rpki-client and the fact that it was so incredibly slow. Since Bob beck@ was around I started to ask him innocent X509 questions ... as if there are innocent X509 questions! Mainly about the abuse of the X509_STORE in rpki-client. Pretty soon it was clear that rpki-client did it all wrong and most of the X509 verification had to be rewritten. Instead of only storing the root certificates in the store and passing the intermediate certs as a chain to the verification function rpki-client threw everything into it. The X509_STORE is just not built for such an abuse and so it was no wonder that this was slow.
  • Lucky me I pulled benno@ with me into this dark hole of libcrypto code. He managed to build up an initial diff to pass the chains as a STACK_OF(X509) and together we managed to get it working. A big thanks goes to ingo@ who documented most of the functions we had to use. Have a look at STACK_OF(3) and sk_pop_free(3) to understand why benno@ and I slowly turned crazy.
  • Our next challenge was to only load the necessary certificate revocation list into the X509_STORE_CTX. While doing those changes it became obvious that some of the data structures needed better lookup functions. Looking up certificates was done using a linear lookup and so we replaced the internal certificate and CRL tables with RB trees for fast lookups. deraadt@ also joined the rpki-client commit fest and changed the output code to use rename(2) so that files are replaced in an atomic operation. Thanks to this rpki-client can now be safely run from cron (there is an example in the default crontab).
  • I did not plan to spend most of my week hacking on rpki-client but in the end I'm happy that I did and the result is fairly impressive. Working with libcrypto code and especially X509 was less than pleasant. Our screams of agony died away in the snowy rocky mountains and made Bob deep dive into UVM with a smile since he knew that benno@ and I had it worse.
  • In case you wonder thanks to all changes at e2k19 rpki-client improved from over 20min run time to validate all VRPS to roughly 1min to do the same job. A factor 20 improvement!
  • Thanks to Theo, Bob and Howie to make this possible. To all the cooks for the great food and to Xplornet for providing us with Internet at the hut.

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Your browser does not support the HTML5 video tag.

Special Guest: Mariusz Zaborski.

Dec 26 2019

1hr 15mins

Play

329: Lucas’ Arts

Podcast cover
Read more

In this episode, we interview Michael W. Lucas about his latest book projects, including the upcoming SNMP Mastery book.

Interview - Michael Lucas

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

Special Guest: Michael W Lucas.

Dec 19 2019

51mins

Play

328: EPYC Netflix Stack

Podcast cover
Read more

LLDB Threading support now ready, Multiple IPSec VPN tunnels with FreeBSD, Netflix Optimized FreeBSD's Network Stack More Than Doubled AMD EPYC Performance, happy eyeballs with unwind(8), AWS got FreeBSD ARM 12, OpenSSH U2F/FIDO support, and more.

Headlines

LLDB Threading support now ready for mainline

Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages.

In February, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support, extending NetBSD's ptrace interface to cover more register types and fix compat32 issues and fixing watchpoint support. Then, I've started working on improving thread support which is taking longer than expected. You can read more about that in my September 2019 report.

So far the number of issues uncovered while enabling proper threading support has stopped me from merging the work-in-progress patches. However, I've finally reached the point where I believe that the current work can be merged and the remaining problems can be resolved afterwards. More on that and other LLVM-related events happening during the last month in this report.

Multiple IPSec VPN tunnels with FreeBSD

The FreeBSD handbook describes an IPSec VPN tunnel between 2 FreeBSD hosts (see https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/ipsec.html)

But it is also possible to have multiple, 2 or more, IPSec VPN tunnels created and running on a FreeBSD host. How to implement and configure this is described below.

The requirements is to have 3 locations (A, B and C) connected with IPSec VPN tunnels using FreeBSD (11.3-RELEASE).

Each location has 1 IPSec VPN host running FreeBSD (VPN host A, B and C).

VPN host A has 2 IPSec VPN tunnels: 1 to location B (VPN host B) and 1 to location C (VPN host C).

News Roundup

Netflix Optimized FreeBSD's Network Stack More Than Doubled AMD EPYC Performance

Drew Gallatin of Netflix presented at the recent EuroBSDcon 2019 conference in Norway on the company's network stack optimizations to FreeBSD. Netflix was working on being able to deliver 200Gb/s network performance for video streaming out of Intel Xeon and AMD EPYC servers, to which they are now at 190Gb/s+ and in the process that doubled the potential of EPYC Naples/Rome servers and also very hefty upgrades too for Intel.

Netflix has long been known to be using FreeBSD in their data centers particularly where network performance is concerned. But in wanting to deliver 200Gb/s throughput from individual servers led them to making NUMA optimizations to the FreeBSD network stack. Allocating NUMA local memory for kernel TLS crypto buffers and for backing files sent via sentfile were among their optimizations. Changes to network connection handling and dealing with incoming connections to Nginx were also made.

For those just wanting the end result, Netflix's NUMA optimizations to FreeBSD resulted in their Intel Xeon servers going from 105Gb/s to 191Gb/s while the NUMA fabric utilization dropped from 40% to 13%.

unwind(8); "happy eyeballs"

In case you are wondering why happy eyeballs: It's a variation on this:
https://en.wikipedia.org/wiki/Happy_Eyeballs

unwind has a concept of a best nameserver type. It considers a configured DoT nameserver to be better than doing it's own recursive resolving. Recursive resolving is considered to be better than asking the dhcp provided nameservers.

This diff sorts the nameserver types by quality, as above (validation, resolving, dead...), and as a tie breaker it adds the median of the round trip time of previous queries into the mix.

One other interesting thing about this is that it gets us past captive portals without a check URL, that's why this diff is so huge, it rips out all the captive portal stuff (please apply with patch -E):
17 files changed, 385 insertions(+), 1683 deletions(-)

Please test this. I'm particularly interested in reports from people who move between networks and need to get past captive portals.

Amazon now has FreeBSD ARM 12

Product Overview

FreeBSD is an operating system used to power servers, desktops, and embedded systems. Derived from BSD, the version of UNIX developed at the University of California, Berkeley, FreeBSD has been continually developed by a large community for more than 30 years.

FreeBSD's networking, security, storage, and monitoring features, including the pf firewall, the Capsicum and CloudABI capability frameworks, the ZFS filesystem, and the DTrace dynamic tracing framework, make FreeBSD the platform of choice for many of the busiest web sites and most pervasive embedded networking and storage systems.

OpenSSH U2F/FIDO support in base

I just committed all the dependencies for OpenSSH security key (U2F) support to base and tweaked OpenSSH to use them directly. This means there will be no additional configuration hoops to jump through to use U2F/FIDO2 security keys.

Hardware backed keys can be generated using "ssh-keygen -t ecdsa-sk" (or "ed25519-sk" if your token supports it). Many tokens require to be touched/tapped to confirm this step.

You'll get a public/private keypair back as usual, except in this case, the private key file does not contain a highly-sensitive private key but instead holds a "key handle" that is used by the security key to derive the real private key at signing time.

So, stealing a copy of the private key file without also stealing your security key (or access to it) should not give the attacker anything.

Once you have generated a key, you can use it normally - i.e. add it to an agent, copy it to your destination's authorized_keys files (assuming they are running -current too), etc. At authentication time, you will be prompted to tap your security key to confirm the signature operation - this makes theft-of-access attacks against security keys more difficult too.

Please test this thoroughly - it's a big change that we want to have stable before the next release.

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Your browser does not support the HTML5 video tag.

Sponsored By:

Dec 12 2019

57mins

Play

327: ZFS Rename Repo

Podcast cover
Read more

We read FreeBSD’s third quarterly status report, OpenBSD on Sparc64, ZoL repo move to OpenZFS, GEOM NOP, keeping NetBSD up-to-date, and more.

Headlines

FreeBSD third quarterly status report for 2019

This quarter the reports team has been more active than usual thanks to a better organization: calls for reports and reminders have been sent regularly, reports have been reviewed and merged quickly (I would like to thank debdrup@ in particular for his reviewing work).

Efficiency could still be improved with the help of our community. In particular, the quarterly team has found that many reports have arrived in the last days before the deadline or even after. I would like to invite the community to follow the guidelines below that can help us sending out the reports sooner.

Starting from next quarter, all quarterly status reports will be prepared the last month of the quarter itself, instead of the first month after the quarter's end. This means that deadlines for submitting reports will be the 1st of January, April, July and October.

Next quarter will then be a short one, covering the months of November and December only and the report will probably be out in mid January.

OpenBSD on Sparc64

OpenBSD, huh? Yes, I usually write about FreeBSD and that’s in fact what I tried installing on the machine first. But I ran into problems with it very early on (never even reached single user mode) and put it aside for later. Since I powered up the SunFire again last month, I needed an OS now and chose OpenBSD for the simple reason that I have it available.

First I wanted to call this article simply “OpenBSD on SPARC” – but that would have been misleading since OpenBSD used to support 32-bit SPARC processors, too. The platform was just put to rest after the 5.9 release.

Version 6.0 was the last release of OpenBSD that came on CD-ROM. When I bought it, I thought that I’d never use the SPARC CD. But here was the chance! While it is an obsolete release, it comes with the cryptographic signatures to verify the next release. So the plan is to start at 6.0 as I can trust the original CDs and then update to the latest release. This will also be an opportunity to recap on some of the things that changed over the various versions.

News Roundup

ZoL repo move to OpenZFS

Because it will contain the ZFS source code for both Linux and FreeBSD, we will rename the "ZFSonLinux" code repository to "OpenZFS". Specifically, the repo at http://github.com/ZFSonLinux/zfs will be moved to the OpenZFS organization, at http://github.com/OpenZFS/zfs.

The next major release of ZFS for Linux and FreeBSD will be "OpenZFS 2.0", and is expected to ship in 2020.

Mcclure111 Sun Thread

A long time ago— like 15 years ago— I worked at Sun Microsystems. The company was nearly dead at the time (it died a couple years later) because they didn't make anything that anyone wanted to buy anymore. So they had a lot of strange ideas about how they'd make their comeback.

GEOM NOP

Sometimes while testing file systems or applications you want to simulate some errors on the disk level. The first time I heard about this need was from Baptiste Daroussin during his presentation at AsiaBSDCon 2016. He mentioned how they had built a test lab with it. The same need was recently discussed during the PGCon 2019, to test a PostgreSQL instance. If you are FreeBSD user, I have great news for you: there is a GEOM provider which allows you to simulate a failing device.

GNOP allows us to configure transparent providers from existing ones. The first interesting option of it is that we can slice the device into smaller pieces, thanks to the ‘offset option’ and ‘stripsesize’. This allows us to observe how the data on the disk is changing. Let’s assume that we want to observe the changes in the GPT table when the GPT flags are added or removed (for example the bootme flags which are described here). We can use dd every time and analyze it using absolute values from the disks.

Keeping NetBSD up-to-date with pkg_comp 2.0

This is a tutorial to guide you through the shiny new pkg_comp 2.0 on NetBSD.

Goals: to use pkg_comp 2.0 to build a binary repository of all the packages you are interested in; to keep the repository fresh on a daily basis; and to use that repository with pkgin to maintain your NetBSD system up-to-date and secure.

This tutorial is specifically targeted at NetBSD but should work on other platforms with some small changes. Expect, at the very least, a macOS-specific tutorial as soon as I create a pkg_comp standalone installer for that platform.

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

Dec 05 2019

1hr 23mins

Play

326: Certified BSD

Podcast cover
Read more

LPI releases BSD Certification, openzfs trip report, Using FreeBSD with ports, LLDB threading support ready, Linux versus Open Source Unix, and more.

Headlines

Linux Professional Institute Releases BSD Specialist Certification - re BSD Certification Group

Linux Professional Institute extends its Open Technology certification track with the BSD Specialist Certification. Starting October 30, 2019, BSD Specialist exams will be globally available. The certification was developed in collaboration with the BSD Certification Group which merged with Linux Professional Institute in 2018.

G. Matthew Rice, the Executive Director of Linux Professional Institute says that "the release of the BSD Specialist certification marks a major milestone for Linux Professional Institute. With this new credential, we are reaffirming our belief in the value of, and support for, all open source technologies. As much as possible, future credentials and educational programs will include coverage of BSD.”

OpenZFS Trip Report

The seventh annual OpenZFS Developer Summit took place on November 4th and 5th in San Francisco and brought together a healthy mix of familiar faces and new community participants. Several folks from iXsystems took part in the talks, hacking, and socializing at this amazing annual event. The messages of the event can be summed up as Unification, Refinement, and Ecosystem Tooling.

News Roundup

Using FreeBSD with Ports (2/2): Tool-assisted updating

In the previous post I explained why sometimes building your software from ports may make sense on FreeBSD. I also introduced the reader to the old-fashioned way of using tools to make working with ports a bit more convenient.

In this follow-up post we’re going to take a closer look at portmaster and see how it especially makes updating from ports much, much easier. For people coming here without having read the previous article: What I describe here is not what every FreeBSD admin today should consider good practice (any more)! It can still be useful in special cases, but my main intention is to discuss this for building up the foundation for what you actually should do today.

LLDB Threading support now ready for mainline

Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages.

In February, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support, extending NetBSD's ptrace interface to cover more register types and fix compat32 issues and fixing watchpoint support. Then, I've started working on improving thread support which is taking longer than expected. You can read more about that in my September 2019 report.

So far the number of issues uncovered while enabling proper threading support has stopped me from merging the work-in-progress patches. However, I've finally reached the point where I believe that the current work can be merged and the remaining problems can be resolved afterwards. More on that and other LLVM-related events happening during the last month in this report.

Linux VS open source UNIX

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

Sponsored By:

Nov 28 2019

1hr

Play

325: Cracking Rainbows

Podcast cover
Read more

FreeBSD 12.1 is here, A history of Unix before Berkeley, FreeBSD development setup, HardenedBSD 2019 Status Report, DNSSEC, compiling RainbowCrack on OpenBSD, and more.

Headlines

FreeBSD 12.1

  • Some of the highlights:

    • BearSSL has been imported to the base system.
    • The clang, llvm, lld, lldb, compiler-rt utilities and libc++ have been updated to version 8.0.1.
    • OpenSSL has been updated to version 1.1.1d.
    • Several userland utility updates.
  • For a complete list of new features and known problems, please see the online release notes and errata list, available at: https://www.FreeBSD.org/releases/12.1R/relnotes.html

A History of UNIX before Berkeley: UNIX Evolution: 1975-1984.

Nobody needs to be told that UNIX is popular today. In this article we will show you a little of where it was yesterday and over the past decade. And, without meaning in the least to minimise the incredible contributions of Ken Thompson and Dennis Ritchie, we will bring to light many of the others who worked on early versions, and try to show where some of the key ideas came from, and how they got into the UNIX of today.

Our title says we are talking about UNIX evolution. Evolution means different things to different people. We use the term loosely, to describe the change over time among the many different UNIX variants in use both inside and outside Bell Labs. Ideas, code, and useful programs seem to have made their way back and forth - like mutant genes - among all the many UNIXes living in the phone company over the decade in question.

Part One looks at some of the major components of the current UNIX system - the text formatting tools, the compilers and program development tools, and so on. Most of the work described in Part One took place at Research'', a part of Bell Laboratories (now AT&T Bell Laboratories, then as nowthe Labs''), and the ancestral home of UNIX. In planned (but not written) later parts, we would have looked at some of the myriad versions of UNIX - there are far more than one might suspect. This includes a look at Columbus and USG and at Berkeley Unix. You'll begin to get a glimpse inside the history of the major streams of development of the system during that time.

News Roundup

My FreeBSD Development Setup

I do my FreeBSD development using git, tmux, vim and cscope.

I keep a FreeBSD fork on my github, I have forked https://github.com/freebsd/freebsd to https://github.com/adventureloop/freebsd

OPNsense 19.7.6 released

As we are experiencing the Suricata community first hand in Amsterdam we thought to release this version a bit earlier than planned. Included is the latest Suricata 5.0.0 release in the development version. That means later this November we will releasing version 5 to the production version as we finish up tweaking the integration and maybe pick up 5.0.1 as it becomes available.

LDAP TLS connectivity is now integrated into the system trust store, which ensures that all required root and intermediate certificates will be seen by the connection setup when they have been added to the authorities section. The same is true for trusting self-signed certificates. On top of this, IPsec now supports public key authentication as contributed by Pascal Mathis.

HardenedBSD November 2019 Status Report.

We at HardenedBSD have a lot of news to share. On 05 Nov 2019, Oliver Pinter resigned amicably from the project. All of us at HardenedBSD owe Oliver our gratitude and appreciation. This humble project, named by Oliver, was born out of his thesis work and the collaboration with Shawn Webb. Oliver created the HardenedBSD repo on GitHub in April 2013. The HardenedBSD Foundation was formed five years later to carry on this great work.

DNSSEC enabled in default unbound(8) configuration.

DNSSEC validation has been enabled in the default unbound.conf(5) in -current. The relevant commits were from Job Snijders (job@)

How to Install Shopware with NGINX and Let's Encrypt on FreeBSD 12

Shopware is the next generation of open source e-commerce software. Based on bleeding edge technologies like Symfony 3, Doctrine2 and Zend Framework Shopware comes as the perfect platform for your next e-commerce project. This tutorial will walk you through the Shopware Community Edition (CE) installation on FreeBSD 12 system by using NGINX as a web server.

  • Requirements

Make sure your system meets the following minimum requirements:

  • Linux-based operating system with NGINX or Apache 2.x (with mod_rewrite) web server installed.
  • PHP 5.6.4 or higher with ctype, gd, curl, dom, hash, iconv, zip, json, mbstring, openssl, session, simplexml, xml, zlib, fileinfo, and pdo/mysql extensions. PHP 7.1 or above is strongly recommended.
  • MySQL 5.5.0 or higher.
  • Possibility to set up cron jobs.
  • Minimum 4 GB available hard disk space.
  • IonCube Loader version 5.0.0 or higher (optional).

How to Compile RainbowCrack on OpenBSD

Project RainbowCrack was originally Zhu Shuanglei's implementation, it's not clear to me if the project is still just his or if it's even been maintained for a while. His page seems to have been last updated in August 2007.

The Project RainbowCrack web page now has just binaries for Windows XP and Linux, both 32-bit and 64-bit versions.

Earlier versions were available as source code. The version 1.2 source code does not compile on OpenBSD, and in my experience it doesn't compile on Linux, either. It seems to date from 2004 at the earliest, and I think it makes some version-2.4 assumptions about Linux kernel headers.

  • You might also look at ophcrack, a more modern tool, although it seems to be focused on cracking Windows XP/Vista/7/8/10 password hashes

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

Nov 21 2019

57mins

Play

324: Emergency Space Mode

Podcast cover
Read more

Migrating drives and zpool between hosts, OpenBSD in 2019, Dragonfly’s new zlib and dhcpcd, Batch renaming images and resolution with awk, a rant on the X11 ICCCM selection system, hammer 2 emergency space mode, and more.

Headlines

Migrating drives and the zpool from one host to another.

Today is the day.

Today I move a zpool from an R710 into an R720. The goal: all services on that zpool start running on the new host.

Fortunately, that zpool is dedicated to jails, more or less. I have done some planning about this, including moving a poudriere on the R710 into a jail.

Now it is almost noon on Saturday, I am sitting in the basement (just outside the server room), and I’m typing this up.

OpenBSD in 2019

I’ve used OpenBSD on and off since 2.1. More back then than in the last 10 years or so though, so I thought I’d try it again.

What triggered this was me finding a silly bug in GNU cpio that has existed with a “FIXME” comment since at least 1994. I checked OpenBSD to see if it had a related bug, but as expected no it was just fine.

I don’t quite remember why I stopped using OpenBSD for servers, but I do remember filesystem corruption on “unexpected power disconnections” (even with softdep turned on), which I’ve never really seen on Linux.

That and that fewer things “just worked” than with Linux, which matters more when I installed more random things than I do now. I’ve become a lot more minimalist. Probably due to less spare time. Life is better when you don’t run things like PHP (not that OpenBSD doesn’t support PHP, just an example) or your own email server with various antispam tooling, and other things.

This is all experience from running OpenBSD on a server. On my next laptop I intend to try running OpenBSD on the dektop, and will see if that more ad-hoc environment works well. E.g. will gnuradio work? Lack of other-OS VM support may be a problem.

  • Verdict

Ouch, that’s a long list of bad stuff. Still, I like it. I’ll continue to run it, and will make sure my stuff continues working on OpenBSD.

And maybe in a year I’ll have a review of OpenBSD on a laptop.

News Roundup

New zlib, new dhcpcd

zlib and dhcpcd are both updated in DragonFly… but my quick perusal of the commits makes it sound like bugfix only; no usage changes needed.

Batch renaming images, including image resolution, with awk

The most recent item on my list of “Geeky things I did that made me feel pretty awesome” is an hour’s adventure that culminated in this code:

$ file IMG* | awk 'BEGIN{a=0} {print substr($1, 1, length($1)-5),a++"_"substr($8,1, length($8)-1)}' | while read fn fr; do echo $(rename -v "s/$fn/img_$fr/g" *); done IMG_20170808_172653_425.jpg renamed as img_0_4032x3024.jpg IMG_20170808_173020_267.jpg renamed as img_1_3024x3506.jpg IMG_20170808_173130_616.jpg renamed as img_2_3024x3779.jpg IMG_20170808_173221_425.jpg renamed as img_3_3024x3780.jpg IMG_20170808_173417_059.jpg renamed as img_4_2956x2980.jpg IMG_20170808_173450_971.jpg renamed as img_5_3024x3024.jpg IMG_20170808_173536_034.jpg renamed as img_6_4032x3024.jpg IMG_20170808_173602_732.jpg renamed as img_7_1617x1617.jpg IMG_20170808_173645_339.jpg renamed as img_8_3024x3780.jpg IMG_20170909_170146_585.jpg renamed as img_9_3036x3036.jpg IMG_20170911_211522_543.jpg renamed as img_10_3036x3036.jpg IMG_20170913_071608_288.jpg renamed as img_11_2760x2760.jpg IMG_20170913_073205_522.jpg renamed as img_12_2738x2738.jpg // ... etc etc

The last item on the aforementioned list is “TODO: come up with a shorter title for this list.”

I hate the X11 ICCCM selection system, and you should too - A Rant

d00d, that document is devilspawn. I've recently spent my nights in pain
implementing the selection mechanism. WHY OH WHY OH WHY? why me? why did I choose to do this? and what sick evil twisted mind wrote this damn spec? I don't know why I'm working with it, I just wanted to make a useful program.

I didn't know what I was getting myself in to. Nobody knows until they try it. And once you start, you're unable to stop. You can't stop, if you stop then you haven't completed it to spec. You can't fail on this, it's just a few pages of text, how can that be so hard? So what if they use Atoms for everything. So what if there's no explicit correlation between the target type of a SelectionNotify event and the type of the property it indicates?

So what if the distinction is ambiguous? So what if the document is littered with such atrocities? It's not the spec's fault, the spec is authoritative. It's obviously YOUR (the implementor's) fault for misunderstanding it. If you didn't misunderstand it, you wouldn't be here complaining about it would you?

HAMMER2 emergency space mode

As anyone who has been running HAMMER1 or HAMMER2 has noticed, snapshots and copy on write and infinite history can eat a lot of disk space, even if the actual file volume isn’t changing much. There’s now an ‘emergency mode‘ for HAMMER2, where disk operations can happen even if there isn’t space for the normal history activity. It’s dangerous, in that the normal protections against data loss if power is cut go away, and snapshots created while in this mode will be mangled. So definitely don’t leave it on!

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Your browser does not support the HTML5 video tag.

Nov 14 2019

46mins

Play

323: OSI Burrito Guy

Podcast cover
Read more

The earliest Unix code, how to replace fail2ban with blacklistd, OpenBSD crossed 400k commits, how to install Bolt CMS on FreeBSD, optimized hammer2, appeasing the OSI 7-layer burrito guys, and more.

Headlines

The Earliest Unix Code: An Anniversary Source Code Release

What is it that runs the servers that hold our online world, be it the web or the cloud? What enables the mobile apps that are at the center of increasingly on-demand lives in the developed world and of mobile banking and messaging in the developing world? The answer is the operating system Unix and its many descendants: Linux, Android, BSD Unix, MacOS, iOS—the list goes on and on. Want to glimpse the Unix in your Mac? Open a Terminal window and enter “man roff” to view the Unix manual entry for an early text formatting program that lives within your operating system.

2019 marks the 50th anniversary of the start of Unix. In the summer of 1969, that same summer that saw humankind’s first steps on the surface of the Moon, computer scientists at the Bell Telephone Laboratories—most centrally Ken Thompson and Dennis Ritchie—began the construction of a new operating system, using a then-aging DEC PDP-7 computer at the labs.

This man sent the first online message 50 years ago

  • As many of you have heard in the past, the first online message ever sent between two computers was "lo", just over 50 years ago, on Oct. 29, 1969.

It was supposed to say "log," but the computer sending the message — based at UCLA — crashed before the letter "g" was typed. A computer at Stanford 560 kilometres away was supposed to fill in the remaining characters "in," as in "log in."

  • The CBC Radio show, “The Current” has a half-hour interview with the man who sent that message, Leonard Kleinrock, distinguished professor of computer science at UCLA

"The idea of the network was you could sit at one computer, log on through the network to a remote computer and use its services there,"

50 years later, the internet has become so ubiquitous that it has almost been rendered invisible. There's hardly an aspect in our daily lives that hasn't been touched and transformed by it.

Q: Take us back to that day 50 years ago. Did you have the sense that this was going to be something you'd be talking about a half a century later?

A: Well, yes and no. Four months before that message was sent, there was a press release that came out of UCLA in which it quotes me as describing what my vision for this network would become. Basically what it said is that this network would be always on, always available. Anybody with any device could get on at anytime from any location, and it would be invisible.

Well, what I missed ... was that this is going to become a social network. People talking to people. Not computers talking to computers, but [the] human element.

Q: Can you briefly explain what you were working on in that lab? Why were you trying to get computers to actually talk to one another?

A: As an MIT graduate student, years before, I recognized I was surrounded by computers and I realized there was no effective [or efficient] way for them to communicate. I did my dissertation, my research, on establishing a mathematical theory of how these networks would work. But there was no such network existing. AT&T said it won't work and, even if it does, we want nothing to do with it.

So I had to wait around for years until the Advanced Research Projects Agency within the Department of Defence decided they needed a network to connect together the computer scientists they were supervising and supporting.

Q: For all the promise of the internet, it has also developed some dark sides that I'm guessing pioneers like yourselves never anticipated.

A: We did not. I knew everybody on the internet at that time, and they were all well-behaved and they all believed in an open, shared free network. So we did not put in any security controls.

When the first spam email occurred, we began to see the dark side emerge as this network reached nefarious people sitting in basements with a high-speed connection, reaching out to millions of people instantaneously, at no cost in time or money, anonymously until all sorts of unpleasant events occurred, which we called the dark side.

But in those early days, I considered the network to be going through its teenage years. Hacking to spam, annoying kinds of effects. I thought that one day this network would mature and grow up. Well, in fact, it took a turn for the worse when nation states, organized crime and extremists came in and began to abuse the network in severe ways.

Q: Is there any part of you that regrets giving birth to this?

A: Absolutely not. The greater good is much more important.

News Roundup

How to use blacklistd(8) with NPF as a fail2ban replacement

blacklistd(8) provides an API that can be used by network daemons to communicate with a packet filter via a daemon to enforce opening and closing ports dynamically based on policy.

The interface to the packet filter is in /libexec/blacklistd-helper (this is currently designed for npf) and the configuration file (inspired from inetd.conf) is in etc/blacklistd.conf

Now, blacklistd(8) will require bpfjit(4) (Just-In-Time compiler for Berkeley Packet Filter) in order to properly work, in addition to, naturally, npf(7) as frontend and syslogd(8), as a backend to print diagnostic messages. Also remember npf shall rely on the npflog* virtual network interface to provide logging for tcpdump() to use.

Unfortunately (dont' ask me why ??) in 8.1 all the required kernel components are still not compiled by default in the GENERIC kernel (though they are in HEAD), and are rather provided as modules. Enabling NPF and blacklistd services would normally result in them being automatically loaded as root, but predictably on securelevel=1 this is not going to happen.

OpenBSD crossed 400,000 commits

Sometime in the last week OpenBSD crossed 400,000 commits (*) upon all our repositories since starting at 1995/10/18 08:37:01 Canada/Mountain. That's a lot of commits by a lot of amazing people.

(*) by one measure. Since the repository is so large and old, there are a variety of quirks including ChangeLog missing entries and branches not convertible to other repo forms, so measuring is hard. If you think you've got a great way of measuring, don't be so sure of yourself -- you may have overcounted or undercounted.

  • Subject to the notes Theo made about under and over counting, FreeBSD should hit 1 million commits (base + ports + docs) some time in 2020
  • NetBSD + pkgsrc are approaching 600,000, but of course pkgsrc covers other operating systems too

How to Install Bolt CMS with Nginx and Let's Encrypt on FreeBSD 12

Bolt is a sophisticated, lightweight and simple CMS built with PHP. It is released under the open-source MIT-license and source code is hosted as a public repository on Github. A bolt is a tool for Content Management, which strives to be as simple and straightforward as possible. It is quick to set up, easy to configure, uses elegant templates. Bolt is created using modern open-source libraries and is best suited to build sites in HTML5 with modern markup. In this tutorial, we will go through the Bolt CMS installation on FreeBSD 12 system by using Nginx as a web server, MySQL as a database server, and optionally you can secure the transport layer by using acme.sh client and Let's Encrypt certificate authority to add SSL support.

  • Requirements
  • The system requirements for Bolt are modest, and it should run on any fairly modern web server:
    • PHP version 5.5.9 or higher with the following common PHP extensions: pdo, mysqlnd, pgsql, openssl, curl, gd, intl, json, mbstring, opcache, posix, xml, fileinfo, exif, zip.
    • Access to SQLite (which comes bundled with PHP), or MySQL or PostgreSQL.
    • Apache with mod_rewrite enabled (.htaccess files) or Nginx (virtual host configuration covered below).
    • A minimum of 32MB of memory allocated to PHP.

hammer2 - Optimize hammer2 support threads and dispatch

Refactor the XOP groups in order to be able to queue strategy calls, whenever possible, to the same CPU as the issuer. This optimizes several cases and reduces unnecessary IPI traffic between cores. The next best thing to do would be to not queue certain XOPs to an H2 support thread at all, but I would like to keep the threads intact for later clustering work.

The best scaling case for this is when one has a large number of user threads doing I/O. One instance of a single-threaded program on an otherwise idle machine might see a slightly reduction in performance but at the same time we completely avoid unnecessarily spamming all cores in the system on the behalf of a single program, so overhead is also significantly lower.

This will tend to increase the number of H2 support threads since we need a certain degree of multiplication for domain separation.

This should significantly increase I/O performance for multi-threaded workloads.

You know, we might as well just run every network service over HTTPS/2 and build another six layers on top of that to appease the OSI 7-layer burrito guys

I've seen the writing on the wall, and while for now you can configure Firefox not to use DoH, I'm not confident enough to think it will remain that way. To that end, I've finally set up my own DoH server for use at Chez Boca. It only involved setting up my own CA to generate the appropriate certificates, install my CA certificate into Firefox, configure Apache to run over HTTP/2 (THANK YOU SO VERY XXXXX­XX MUCH GOOGLE FOR SHOVING THIS HTTP/2 XXXXX­XXX DOWN OUR THROATS!—no, I'm not bitter) and write a 150 line script that just queries my own local DNS, because, you know, it's more XXXXX­XX secure or some XXXXX­XXX reason like that.

Sigh.

Beastie Bits

Feedback/Questions

Your browser does not support the HTML5 video tag.

Nov 07 2019

49mins

Play

322: Happy Birthday, Unix

Podcast cover
Read more

Unix is 50, Hunting down Ken's PDP-7, OpenBSD and OPNSense have new releases, Clarification on what GhostBSD is, sshuttle - VPN over SSH, and more.

Headlines

Unix is 50

In the summer of 1969 computer scientists Ken Thompson and Dennis Ritchie created the first implementation of Unix with the goal of designing an elegant and economical operating system for a little-used PDP-7 minicomputer at Bell Labs. That modest project, however, would have a far-reaching legacy. Unix made large-scale networking of diverse computing systems — and the Internet — practical. The Unix team went on to develop the C language, which brought an unprecedented combination of efficiency and expressiveness to programming. Both made computing more "portable". Today, Linux, the most popular descendent of Unix, powers the vast majority of servers, and elements of Unix and Linux are found in most mobile devices. Meanwhile C++ remains one of the most widely used programming languages today. Unix may be a half-century old but its influence is only growing.

Hunting down Ken's PDP-7: video footage found

In my prior blog post, I traced Ken's scrounged PDP-7 to SN 34. In this post I'll show that we have actual video footage of that PDP-7 due to an old film from Bell Labs. this gives us almost a minute of footage of the PDP-7 Ken later used to create Unix.

News Roundup

OpenBSD 6.6 Released

OPNsense 19.7.5 released

Hello friends and followers, Lots of plugin and ports updates this time with a few minor improvements in all core areas. Behind the scenes we are starting to migrate the base system to version

12.1 which is supposed to hit the next 20.1 release. Stay tuned for more infos in the next month or so.

Here are the full patch notes:

  • system: show all swap partitions in system information widget
  • system: flatten services_get() in preparation for removal
  • system: pin Syslog-ng version to specific package name
  • system: fix LDAP/StartTLS with user import page
  • system: fix a PHP warning on authentication server page
  • system: replace most subprocess.call use
  • interfaces: fix devd handling of carp devices (contributed by stumbaumr)
  • firewall: improve firewall rules inline toggles
  • firewall: only allow TCP flags on TCP protocol
  • firewall: simplify help text for direction setting
  • firewall: make protocol log summary case insensitive
  • reporting: ignore malformed flow records
  • captive portal: fix type mismatch for timeout read
  • dhcp: add note for static lease limitation with lease registration (contributed by Northguy)
  • ipsec: add margintime and rekeyfuzz options
  • ipsec: clear $dpdline correctly if not set
  • ui: fix tokenizer reorder on multiple saves
  • plugins: os-acme-client 1.26[1]
  • plugins: os-bind will reload bind on record change (contributed by blablup)
  • plugins: os-etpro-telemetry minor subprocess.call replacement
  • plugins: os-freeradius 1.9.4[2]
  • plugins: os-frr 1.12[3]
  • plugins: os-haproxy 2.19[4]
  • plugins: os-mailtrail 1.2[5]
  • plugins: os-postfix 1.11[6]
  • plugins: os-rspamd 1.8[7]
  • plugins: os-sunnyvalley LibreSSL support (contributed by Sunny Valley Networks)
  • plugins: os-telegraf 1.7.6[8]
  • plugins: os-theme-cicada 1.21 (contributed by Team Rebellion)
  • plugins: os-theme-tukan 1.21 (contributed by Team Rebellion)
  • plugins: os-tinc minor subprocess.call replacement
  • plugins: os-tor 1.8 adds dormant mode disable option (contributed by Fabian Franz)
  • plugins: os-virtualbox 1.0 (contributed by andrewhotlab)

Dealing with the misunderstandings of what is GhostBSD

Since the release of 19.09, I have seen a lot of misunderstandings on what is GhostBSD and the future of GhostBSD. GhostBSD is based on TrueOS with FreeBSD 12 STABLE with our twist to it. We are still continuing to use TrueOS for OpenRC, and the new package's system for the base system that is built from ports. GhostBSD is becoming a slow-moving rolling release base on the latest TrueOS with FreeBSD 12 STABLE. When FreeBSD 13 STABLE gets released, GhostBSD will be upgraded to TrueOS with FreeBSD 13 STABLE.

Our official desktop is MATE, which means that the leading developer of GhostBSD does not officially support XFCE. Community releases are maintained by the community and for the community. GhostBSD project will provide help to build and to host the community release. If anyone wants to have a particular desktop supported, it is up to the community. Sure I will help where I can, answer questions and guide new community members that contribute to community release.

There is some effort going on for Plasma5 desktop. If anyone is interested in helping with XFCE and Plasma5 or in creating another community release, you are well come to contribute. Also, Contribution to the GhostBSD base system, to ports and new ports, and in house software are welcome. We are mostly active on Telegram https://t.me/ghostbsd, but you can also reach us on the forum.

SHUTTLE – VPN over SSH | VPN Alternative

Looking for a lightweight VPN client, but are not ready to spend a monthly recurring amount on a VPN? VPNs can be expensive depending upon the quality of service and amount of privacy you want. A good VPN plan can easily set you back by 10$ a month and even that doesn’t guarantee your privacy. There is no way to be sure whether the VPN is storing your confidential information and traffic logs or not. sshuttle is the answer to your problem it provides VPN over ssh and in this article we’re going to explore this cheap yet powerful alternative to the expensive VPNs. By using open source tools you can control your own privacy.

  • VPN over SSH – sshuttle

sshuttle is an awesome program that allows you to create a VPN connection from your local machine to any remote server that you have ssh access on. The tunnel established over the ssh connection can then be used to route all your traffic from client machine through the remote machine including all the dns traffic. In the bare bones sshuttle is just a proxy server which runs on the client machine and forwards all the traffic to a ssh tunnel. Since its open source it holds quite a lot of major advantages over traditional VPN.

OpenSSH 8.1 Released

  • Security

    • ssh(1), sshd(8), ssh-add(1), ssh-keygen(1): an exploitable integer overflow bug was found in the private key parsing code for the XMSS key type. This key type is still experimental and support for it is not compiled by default. No user-facing autoconf option exists in portable OpenSSH to enable it. This bug was found by Adam Zabrocki and reported via SecuriTeam's SSD program.
    • ssh(1), sshd(8), ssh-agent(1): add protection for private keys at rest in RAM against speculation and memory side-channel attacks like Spectre, Meltdown and Rambleed. This release encrypts private keys when they are not in use with a symmetric key that is derived from a relatively large "prekey" consisting of random data (currently 16KB).
  • This release includes a number of changes that may affect existing configurations:

    • ssh-keygen(1): when acting as a CA and signing certificates with an RSA key, default to using the rsa-sha2-512 signature algorithm. Certificates signed by RSA keys will therefore be incompatible with OpenSSH versions prior to 7.2 unless the default is overridden (using "ssh-keygen -t ssh-rsa -s ...").
  • New Features

    • ssh(1): Allow %n to be expanded in ProxyCommand strings
    • ssh(1), sshd(8): Allow prepending a list of algorithms to the default set by starting the list with the '' character, E.g. "HostKeyAlgorithms ssh-ed25519"
    • ssh-keygen(1): add an experimental lightweight signature and verification ability. Signatures may be made using regular ssh keys held on disk or stored in a ssh-agent and verified against an authorized_keys-like list of allowed keys. Signatures embed a namespace that prevents confusion and attacks between different usage domains (e.g. files vs email).
    • ssh-keygen(1): print key comment when extracting public key from a private key.
    • ssh-keygen(1): accept the verbose flag when searching for host keys in known hosts (i.e. "ssh-keygen -vF host") to print the matching host's random-art signature too.
    • All: support PKCS8 as an optional format for storage of private keys to disk. The OpenSSH native key format remains the default, but PKCS8 is a superior format to PEM if interoperability with non-OpenSSH software is required, as it may use a less insecure key derivation function than PEM's.

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

Oct 31 2019

1hr 7mins

Play

321: The Robot OS

Podcast cover
Read more

An interview with Trenton Schulz about his early days with FreeBSD, Robot OS, Qt, and more.

Interview - Trenton Schulz - freenas@norwegianrockcat.com

Robot OS on FreeBSD

  • BR: Welcome to the show. Can you tell us a little bit about yourself and how you got started with BSD?
  • AJ: You were working for Trolltech (creators of Qt). Was FreeBSD used there and how?
  • BR: Can you tell us more about the work you are doing with Robot OS on FreeBSD?
  • AJ: Was EuroBSDcon your first BSD conference? How did you like it?
  • BR: Do you have some tips or advice on how to get started with the BSDs?
  • AJ: Is there anything else you’d like to tell us before we let you go?

Beastie Bits

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

Special Guest: Trenton Shulz.

Oct 24 2019

55mins

Play

320: Codebase: Neck Deep

Podcast cover
Read more

Headlines

FreeBSD and custom firmware on the Google Pixelbook

  • FreeBSD and custom firmware on the Google Pixelbook

Back in 2015, I jumped on the ThinkPad bandwagon by getting an X240 to run FreeBSD on. Unlike most people in the ThinkPad crowd, I actually liked the clickpad and didn\u2019t use the trackpoint much. But this summer I\u2019ve decided that it was time for something newer. I wanted something..

  • lighter and thinner (ha, turns out this is actually important, I got tired of carrying a T H I C C laptop - Apple was right all along);
  • with a 3:2 display (why is Lenovo making these Serious Work\u2122 laptops 16:9 in the first place?? 16:9 is awful in below-13-inch sizes especially);
  • with a HiDPI display (and ideally with a good size for exact 2x scaling instead of fractional);
  • with USB-C ports;
  • without a dGPU, especially without an NVIDIA GPU;
  • assembled with screws and not glue (I don\u2019t necessarily need expansion and stuff in a laptop all that much, but being able to replace the battery without dealing with a glued chassis is good);
  • supported by FreeBSD of course (\u201csome development required\u201d is okay but I\u2019m not going to write big drivers);
  • how about something with open source firmware, that would be fun.

I was considering a ThinkPad X1 Carbon from an old generation - the one from the same year as the X230 is corebootable, so that\u2019s fun. But going back in processor generations just doesn\u2019t feel great. I want something more efficient, not less!

And then I discovered the Pixelbook. Other than the big huge large bezels around the screen, I liked everything about it. Thin aluminum design, a 3:2 HiDPI screen, rubber palm rests (why isn\u2019t every laptop ever doing that?!), the \u201cconvertibleness\u201d (flip the screen around to turn it into.. something rather big for a tablet, but it is useful actually), a Wacom touchscreen that supports a pen, mostly reasonable hardware (Intel Wi-Fi), and that famous coreboot support (Chromebooks\u2019 stock firmware is coreboot + depthcharge).

So here it is, my new laptop, a Google Pixelbook.

  • Conclusion

Pixelbook, FreeBSD, coreboot, EDK2 good.

Seriously, I have no big words to say, other than just recommending this laptop to FOSS enthusiasts :)

Porting NetBSD to the AMD x86-64: a case study in OS portability

  • Abstract

NetBSD is known as a very portable operating system, currently running on 44 different architectures (12 different types of CPU). This paper takes a look at what has been done to make it portable, and how this has decreased the amount of effort needed to port NetBSD to a new architecture. The new AMD x86-64 architecture, of which the specifications were published at the end of 2000, with hardware to follow in 2002, is used as an example.

  • Portability

Supporting multiple platforms was a primary goal of the NetBSD project from the start. As NetBSD was ported to more and more platforms, the NetBSD kernel code was adapted to become more portable along the way.

  • General

Generally, code is shared between ports as much as possible. In NetBSD, it should always be considered if the code can be assumed to be useful on other architectures, present or future. If so, it is machine-independent and put it in an appropriate place in the source tree. When writing code that is intended to be machine-independent, and it contains conditional preprocessor statements depending on the architecture, then the code is likely wrong, or an extra abstraction layer is needed to get rid of these statements.

  • Types

Assumptions about the size of any type are not made. Assumptions made about type sizes on 32-bit platforms were a large problem when 64-bit platforms came around. Most of the problems of this kind had to be dealt with when NetBSD was ported to the DEC Alpha in 1994. A variation on this problem had to be dealt with with the UltraSPARC (sparc64) port in 1998, which is 64-bit, but big endian (vs. the little-endianness of the Alpha). When interacting with datastructures of a fixed size, such as on-disk metadata for filesystems, or datastructures directly interpreted by device hardware, explicitly sized types are used, such as uint32_t, int8_t, etc.

  • Conclusions and future work

The port of NetBSD to AMD's x86-64 architecture was done in six weeks, which confirms NetBSD's reputation as being a very portable operating system. One week was spent setting up the cross-toolchain and reading the x86-64 specifications, three weeks were spent writing the kernel code, one week was spent writing the userspace code, and one week testing and debugging it all. No problems were observed in any of the machine-independent parts of the kernel during test runs; all (simulated) device drivers, file systems, etc, worked without modification.

News Roundup

ZFS performance really does degrade as you approach quota limits

Every so often (currently monthly), there is an "OpenZFS leadership meeting". What this really means is 'lead developers from the various ZFS implementations get together to talk about things'. Announcements and meeting notes from these meetings get sent out to various mailing lists, including the ZFS on Linux ones.

  • In the September meeting notes, I read a very interesting (to me) agenda item:

    • Relax quota semantics for improved performance (Allan Jude)
    • Problem: As you approach quotas, ZFS performance degrades.
    • Proposal: Can we have a property like quota-policy=strict or loose, where we can optionally allow ZFS to run over the quota as long as performance is not decreased.

This is very interesting to me because of two reasons. First, in the past we have definitely seen significant problems on our OmniOS machines, both when an entire pool hits a quota limit and when a single filesystem hits a refquota limit. It's nice to know that this wasn't just our imagination and that there is a real issue here. Even better, it might someday be improved (and perhaps in a way that we can use at least some of the time).

Second, any number of people here run very close to and sometimes at the quota limits of both filesystems and pools, fundamentally because people aren't willing to buy more space. We have in the past assumed that this was relatively harmless and would only make people run out of space. If this is a known issue that causes serious performance degradation, well, I don't know if there's anything we can do, but at least we're going to have to think about it and maybe push harder at people. The first step will have to be learning the details of what's going on at the ZFS level to cause the slowdown. (It's apparently similar to what happens when the pool is almost full, but I don't know the specifics of that either.)

With that said, we don't seem to have seen clear adverse effects on our Linux fileservers, and they've definitely run into quota limits (repeatedly). One possible reason for this is that having lots of RAM and SSDs makes the effects mostly go away. Another possible reason is that we haven't been looking closely enough to see that we're experiencing global slowdowns that correlate to filesystems hitting quota limits. We've had issues before with somewhat subtle slowdowns that we didn't understand (cf), so I can't discount that we're having it happen again.

Fixing up KA9Q-unix, or "neck deep in 30 year old codebases.."

I'll preface this by saying - yes, I'm still neck deep in FreeBSD's wifi stack and 802.11ac support, but it turns out it's slow work to fix 15 year old locking related issues that worked fine on 11abg cards, kinda worked ok on 11n cards, and are terrible for these 11ac cards. I'll .. get there.

Anyhoo, I've finally been mucking around with AX.25 packet radio. I've been wanting to do this since I was a teenager and found out about its existence, but back in high school and .. well, until a few years ago really .. I didn't have my amateur radio licence. But, now I do, and I've done a bunch of other stuff with a bunch of other radios. The main stumbling block? All my devices are either Apple products or run FreeBSD - and none of them have useful AX.25 stacks. The main stacks of choice these days run on Linux, Windows or are a full hardware TNC.

So yes, I was avoiding hacking on AX.25 stuff because there wasn't a BSD compatible AX.25 stack. I'm 40 now, leave me be.

But! A few weeks ago I found that someone was still running a packet BBS out of San Francisco. And amazingly, his local node ran on FreeBSD! It turns out Jeremy (KK6JJJ) ported both an old copy of KA9Q and N0ARY-BBS to run on FreeBSD! Cool!

I grabbed my 2m radio (which is already cabled up for digital modes), compiled up his KA9Q port, figured out how to get it to speak to Direwolf, and .. ok. Well, it worked. Kinda.

HAMMER2 and fsck for review

HAMMER2 is Copy on Write, meaning changes are made to copies of existing data. This means operations are generally atomic and can survive a power outage, etc. (You should read up on it!) However, there\u2019s now a fsck command, useful if you want a report of data validity rather than any manual repair process.

[The return of startx(1) for non-root users with some caveats

Mark Kettenis (kettenis@) has recently committed changes which restore a certain amount of startx(1)/xinit(1) functionality for non-root users. The commit messages explain the situation:

CVSROOT: /cvs
Module name: src
Changes by: kettenis@cvs.openbsd.org 2019/09/15 06:25:41

Modified files:
etc/etc.amd64 : fbtab
etc/etc.arm64 : fbtab
etc/etc.hppa : fbtab
etc/etc.i386 : fbtab
etc/etc.loongson: fbtab
etc/etc.luna88k: fbtab
etc/etc.macppc : fbtab
etc/etc.octeon : fbtab
etc/etc.sgi : fbtab
etc/etc.sparc64: fbtab

Log message:
Add ttyC4 to lost of devices to change when logging in on ttyC0 (and in some cases also the serial console) such that X can use it as its VT when running without root privileges.

ok jsg@, matthieu@
CVSROOT: /cvs
Module name: xenocara
Changes by: kettenis@cvs.openbsd.org 2019/09/15 06:31:08

Modified files:
xserver/hw/xfree86/common: xf86AutoConfig.c

Log message:
Add modesetting driver as a fall-back when appropriate such that we can use it when running without root privileges which prevents us from scanning the PCI bus.

This makes startx(1)/xinit(1) work again on modern systems with inteldrm(4), radeondrm(4) and amdgpu(4). In some cases this will result in using a different driver than with xenodm(4) which may expose issues (e.g. when we prefer the intel Xorg driver) or loss of acceleration (e.g. older cards supported by radeondrm(4)).

ok jsg@, matthieu@

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

Oct 17 2019

56mins

Play

319: Lack Rack, Jack

Podcast cover
Read more

Causing ZFS corruption for fun, NetBSD Assembly Programming Tutorial, The IKEA Lack Rack for Servers, a new OmniOS Community Edition LTS has been published, List Block Devices on FreeBSD lsblk(8) Style, Project Trident 19.10 available, and more.

Headlines

Causing ZFS corruption for fun and profit

Datto backs up data, a lot of it. At the time of writing Datto has over 500 PB of data stored on ZFS. This count includes both backup appliances that are sent to customer sites, as well as cloud storage servers that are used for secondary and tertiary backup of those appliances. At this scale drive swaps are a daily occurrence, and data corruption is inevitable. How we handle this corruption when it happens determines whether we truly lose data, or successfully restore from secondary backup. In this post we'll be showing you how at Datto we intentionally cause corruption in our testing environments, to ensure we're building software that can properly handle these scenarios.

  • Causing Corruption

Since this is a mirror setup, a naive solution to cause corruption would be to randomly dd the same sectors of both /dev/sdb and /dev/sdc. This works, but is equally likely to just overwrite random unused space, or take down the zpool entirely. What we really want is to corrupt a specific snapshot, or even a specific file in that snapshot, to simulate a more realistic minor corruption event. Luckily we have a tool called zdb that lets us view some low level information about datasets.

  • Conclusion

At the 500 PB scale, it's not a matter of if data corruption will happen but when. Intentionally causing corruption is one of the strategies we use to ensure we're building software that can handle these rare (but inevitable) events.

To others out there using ZFS: I'm curious to hear how you've solved this problem. We did quite a bit of experimentation with zinject before going with this more brute force method. So I'd be especially interested if you've had luck simply simulating corruption with zinject.

NetBSD Assembly Programming Tutorial

A sparc64 version is also being prepared and will be added when done

This post describes how to write a simple hello world program in pure assembly on NetBSD/amd64. We will not use (nor link against) libc, nor use gcc to compile it. I will be using GNU as (gas), and therefore the AT&T syntax instead of Intel.

  • Why assembly?

Why not? Because it's fun to program in assembly directly. Contrary to a popular belief assembly programs aren't always faster than what optimizing compilers produce. Nevertheless it's good to be able to read assembly, especially when debugging C programs

  • Due to the nature of the guide, visit the site for the complete breakdown

News Roundup

The IKEA Lack Rack for Servers

  • The LackRack

First occurrence on eth0:2010 Winterlan, the LackRack is the ultimate, low-cost, high shininess solution for your modular datacenter-in-the-living-room. Featuring the LACK (side table) from Ikea, the LackRack is an easy-to-implement, exact-fit datacenter building block. It's a little known fact that we have seen Google engineers tinker with Lack tables since way back in 2009.

The LackRack will certainly make its appearance again this summer at eth0:2010 Summer.

  • Summary

When temporarily not in use, multiple LackRacks can be stacked in a space-efficient way without disassembly, unlike competing 19" server racks.

The LackRack was first seen on eth0:2010 Winterlan in the no-shoe Lounge area. Its low-cost and perfect fit are great for mounting up to 8 U of 19" hardware, such as switches (see below), or perhaps other 19" gear. It's very easy to assemble, and thanks to the design, they are stable enough to hold (for example) 19" switches and you can put your bottle of Club-Mate on top! Multi-shiny LackRack can also be painted to your specific preferences and the airflow is unprecedented!

  • Howto

You can find a howto on buying a LackRack on this page. This includes the proof that a 19" switch can indeed be placed in the LackRack in its natural habitat!

OmniOS Community Edition r151030 LTS - Published at May 6, 2019

The OmniOS Community Edition Association is proud to announce the general availability of OmniOS - r151030.

OmniOS is published according to a 6-month release cycle, r151030 LTS takes over from r151028, published in November 2018; and since it is a LTS release it also takes over from r151022. The r151030 LTS release will be supported for 3 Years. It is the first LTS release published by the OmniOS CE Association since taking over the reins from OmniTI in 2017. The next LTS release is scheduled for May 2021. The old stable r151026 release is now end-of-life. See the release schedule for further details.

This is only a small selection of the new features, and bug fixes in the new release; review the release notes for full details.

If you upgrade from r22 and want to see all new features added since then, make sure to also read the release notes for r24, r26 and r28.

List Block Devices on FreeBSD lsblk(8) Style

When I have to work on Linux systems I usually miss many nice FreeBSD tools such as these for example to name the few: sockstat, gstat, top -b -o res, top -m io -o total, usbconfig, rcorder, beadm/bectl, idprio/rtprio,… but sometimes – which rarely happens – Linux has some very useful tool that is not available on FreeBSD. An example of such tool is lsblk(8) that does one thing and does it quite well – lists block devices and their contents. It has some problems like listing a disk that is entirely used under ZFS pool on which lsblk(8) displays two partitions instead of information about ZFS just being there – but we all know how much in some circles the CDDL licensed ZFS is unloved in that GPL world.

Example lsblk(8) output from Linux system:

$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sr0 11:0 1 1024M 0 rom
sda 8:0 0 931.5G 0 disk
|-sda1 8:1 0 500M 0 part /boot
`-sda2 8:2 0 931G 0 part
|-vg_local-lv_root (dm-0) 253:0 0 50G 0 lvm /
|-vg_local-lv_swap (dm-1) 253:1 0 17.7G 0 lvm [SWAP]
`-vg_local-lv_home (dm-2) 253:2 0 1.8T 0 lvm /home
sdc 8:32 0 232.9G 0 disk
`-sdc1 8:33 0 232.9G 0 part
`-md1 9:1 0 232.9G 0 raid10 /data
sdd 8:48 0 232.9G 0 disk
`-sdd1 8:49 0 232.9G 0 part
`-md1 9:1 0 232.9G 0 raid10 /data

What FreeBSD offers in this department? The camcontrol(8) and geom(8) commands are available. You can also use gpart(8) command to list partitions. Below you will find output of these commands from my single disk laptop. Please note that because of WordPress limitations I need to change all > < characters to ] [ ones in the commands outputs.

  • See the article for the rest of the guide

Project Trident 19.10 Now Available

This is a general package update to the CURRENT release repository based upon TrueOS 19.10

  • PACKAGE CHANGES FROM 19.08

    • New Packages: 601
    • Deleted Packages: 165
    • Updated Packages: 3341

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

Oct 10 2019

1hr 7mins

Play

318: The TrueNAS Library

Podcast cover
Read more

DragonFlyBSD vs. FreeBSD vs. Linux benchmark on Ryzen 7, JFK Presidential Library chooses TrueNAS for digital archives, FreeBSD 12.1-beta is available, cool but obscure X11 tools, vBSDcon trip report, Project Trident 12-U7 is available, a couple new Unix artifacts, and more.

Headlines

DragonFlyBSD 5.6 vs. FreeBSD 12 vs. Linux - Ryzen 7 3700X

For those wondering how well FreeBSD and DragonFlyBSD are handling AMD's new Ryzen 3000 series desktop processors, here are some benchmarks on a Ryzen 7 3700X with MSI MEG X570 GODLIKE where both of these popular BSD operating systems were working out-of-the-box. For some fun mid-week benchmarking, here are those results of FreeBSD 12.0 and DragonFlyBSD 5.6.2 up against openSUSE Tumbleweed and Ubuntu 19.04.

Back in July I looked at FreeBSD 12 on the Ryzen 9 3900X but at that time at least DragonFlyBSD had troubles booting on that system. When trying out the Ryzen 7 3700X + MSI GODLIKE X570 motherboard on the latest BIOS, everything "just worked" without any compatibility issues for either of these BSDs.

We've been eager to see how well DragonFlyBSD is performing on these new AMD Zen 2 CPUs with DragonFlyBSD lead developer Matthew Dillon having publicly expressed being impressed by the new AMD Ryzen 3000 series CPUs.

For comparison to those BSDs, Ubuntu 19.04 and openSUSE Tumbleweed were tested on the same hardware in their out-of-the-box configurations. While Clear Linux is normally the fastest, on this system Clear's power management defaults had caused issues in being unable to detect the Samsung 970 EVO Plus NVMe SSD used for testing and so we left it out this round.

All of the hardware was the same throughout testing as were the BIOS settings and running the Ryzen 7 3700X at stock speeds. (Any differences in the reported hardware for the system table just come down to differences in what is exposed by each OS for reporting.) All of the BSD/Linux benchmarks on this eight core / sixteen thread processor were run via the Phoronix Test Suite. In the case of FreeBSD 12.0, we benchmarked both with its default LLVM Clang 6.0 compiler as well as with GCC 9.1 so that it would match the GCC compiler being the default on the other operating systems under test.

JFK Presidential Library Chooses iXsystems TrueNAS to Preserve Precious Digital Archives

iXsystems is honored to have the TrueNAS® M-Series unified storage selected to store, serve, and protect the entire digital archive for the John F. Kennedy Library Foundation. This is in support of the collection at the John F. Kennedy Presidential Library and Museum (JFK Library). Over the next several years, the Foundation hopes to grow the digital collection from hundreds of terabytes today to cover much more of the Archives at the Kennedy Library. Overall there is a total of 25 million documents, audio recordings, photos, and videos once the project is complete.

Having first deployed the TrueNAS M50-HA earlier in 2019, the JFK Library has now completed the migration of its existing digital collection and is now in the process of digitizing much of the rest of its vast collection.

Not only is the catalog of material vast, it is also diverse, with files being copied to the storage system from a variety of sources in numerous file types. To achieve this ambitious goal, the library required a high-end NAS system capable of sharing with a variety of systems throughout the digitization process. The digital archive will be served from the TrueNAS M50 and made available to both in-person and online visitors.

With precious material and information comes robust demands. The highly-available TrueNAS M-Series has multiple layers of protection to help keep data safe, including data scrubs, checksums, unlimited snapshots, replication, and more. TrueNAS is also inherently scalable with data shares only limited by the number of drives connected to the pool. Perfect for archival storage, the deployed TrueNAS M50 will grow with the library’s content, easily expanding its storage capacity over time as needed. Supporting a variety of protocols, multi-petabyte scalability in a single share, and anytime, uninterrupted capacity expansion, the TrueNAS M-Series ticked all the right boxes.

News Roundup

FreeBSD 12.1-beta available

FreeBSD 12.0 is already approaching one year old while FreeBSD 12.1 is now on the way as the next installment with various bug/security fixes and other alterations to this BSD operating system.

FreeBSD 12.1 has many security/bug fixes throughout, no longer enables "-Werror" by default as a compiler flag (Update: This change is just for the GCC 4.2 compiler), has imported BearSSL into the FreeBSD base system as a lightweight TLS/SSL implementation, bzip2recover has been added, and a variety of mostly lower-level changes. More details can be found via the in-progress release notes.

For those with time to test this weekend, FreeBSD 12.1 Beta 1 is available for all prominent architectures.

The FreeBSD release team is planning for at least another beta or two and around three release candidates. If all goes well, FreeBSD 12.1 will be out in early November.

Cool, but obscure X11 tools. More suggestions in the source link

  • ASClock
  • Free42
  • FSV2
  • GLXGears
  • GMixer
  • GVIM
  • Micropolis
  • Sunclock
  • Ted
  • TiEmu
  • X026
  • X48
  • XAbacus
  • XAntfarm
  • XArchiver
  • XASCII
  • XBiff
  • XBill
  • XBoard
  • XCalc
  • XCalendar
  • XCHM
  • XChomp
  • XClipboard
  • XClock
  • XClock/Cat Clock
  • XColorSel
  • XConsole
  • XDiary
  • XEarth
  • XEdit
  • Xev
  • XEyes
  • XFontSel
  • XGalaga
  • XInvaders 3D
  • XKill
  • XLennart
  • XLoad
  • XLock
  • XLogo
  • XMahjongg
  • XMan
  • XMessage
  • XmGrace
  • XMixer
  • XmMix
  • XMore
  • XMosaic
  • XMOTD
  • XMountains
  • XNeko
  • XOdometer
  • XOSView
  • Xplore
  • XPostIt
  • XRoach
  • XScreenSaver
  • XSnow
  • XSpread
  • XTerm
  • XTide
  • Xv
  • Xvkbd
  • XWPE
  • XZoom

vBSDCon 2019 trip report from iXSystems

The fourth biennial vBSDCon was held in Reston, VA on September 5th through 7th and attracted attendees and presenters from not only the Washington, DC area, but also Canada, Germany, Kenya, and beyond. While MeetBSD caters to Silicon Valley BSD enthusiasts on even years, vBSDcon caters to East Coast and DC area enthusiasts on odd years. Verisign was again the key sponsor of vBSDcon 2019 but this year made a conscious effort to entrust the organization of the event to a team of community members led by Dan Langille, who you probably know as the lead BSDCan organizer. The result of this shift was a low key but professional event that fostered great conversation and brainstorming at every turn.

Project Trident 12-U7 now available

A Couple new Unix Artifacts

I fear we're drifting a bit here and the S/N ratio is dropping a bit w.r.t the actual history of Unix. Please no more on the relative merits of version control systems or alternative text processing systems.

So I'll try to distract you by saying this. I'm sitting on two artifacts that have recently been given to me:

  • by two large organisations
  • of great significance to Unix history
  • who want me to keep "mum" about them
  • as they are going to make announcements about them soon*

and I am going slowly crazy as I wait for them to be offically released. Now you have a new topic to talk about :-)

Cheers, Warren

* for some definition of "soon"

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

Oct 03 2019

46mins

Play

317: Bots Building Jails

Podcast cover
Read more

Setting up buildbot in FreeBSD jails, Set up a mail server with OpenSMTPD, Dovecot and Rspamd, OpenBSD amateur packet radio with HamBSD, DragonFlyBSD's HAMMER2 gets fsck, return of startx for users.

Headlines

EuroBSDcon 2019 Recap

We’re back from EuroBSDcon in Lillehammer, Norway. It was a great conference with 212 people attending. 2 days of tutorials, parallel to the FreeBSD Devsummit, followed by two days of talks. Some speakers uploaded their slides to papers.freebsd.org already with more to come.

The social event was also interesting. We visited an open air museum with building preserved from different time periods. In the older section they had a collection of farm buildings, a church originally built in the 1200s and relocated to the museum, and a school house. In the more modern area, they had houses from 1915, and each decade from 1930 to 1990, plus a “house of the future” as imagined in 2001. Many had open doors to allow you to tour the inside, and some were even “inhabited”. The latter fact gave a much more interactive experience and we could learn additional things about the history of that particular house. The town at the end included a general store, a post office, and more. Then, we all had a nice dinner together in the museum’s restaurant.

  • The opening keynote by Patricia Aas was very good. Her talk on embedded ethics, from her perspective as someone trying to defend the sanctity of Norwegian elections, and a former developer for the Opera web browser, provided a great deal of insight into the issues. Her points about how the tech community has unleashed a very complex digital work upon people with barely any technical literacy were well taken. Her stories of trying to explain the problems with involving computers in the election process to journalists and politicians struck a chord with many of us, who have had to deal with legislation written by those who do not truly understand the issues with technology.

Setting up buildbot in FreeBSD jails

In this article, I would like to present a tutorial to set up buildbot, a continuous integration (CI) software (like Jenkins, drone, etc.), making use of FreeBSD’s containerization mechanism "jails". We will cover terminology, rationale for using both buildbot and jails together, and installation steps. At the end, you will have a working buildbot instance using its sample build configuration, ready to play around with your own CI plans (or even CD, it’s very flexible!). Some hints for production-grade installations are given, but the tutorial steps are meant for a test environment (namely a virtual machine). Buildbot’s configuration and detailed concepts are not in scope here.

Setting up a mail server with OpenSMTPD, Dovecot and Rspamd

  • Self-hosting and encouraging smaller providers is for the greater good

First of all, I was not clear enough about the political consequences of centralizing mail services at Big Mailer Corps.

It doesn’t make sense for Random Joe, sharing kitten pictures with his family and friends, to build a personal mail infrastructure when multiple Big Mailer Corps offer “for free” an amazing quality of service. They provide him with an e-mail address that is immediately available and which will generally work reliably. It really doesn’t make sense for Random Joe not to go there, and particularly if even techies go there without hesitation, proving it is a sound choice.

There is nothing wrong with Random Joes using a service that works.

What is terribly wrong though is the centralization of a communication protocol in the hands of a few commercial companies, EVERY SINGLE ONE OF THEM coming from the same country (currently led by a lunatic who abuses power and probably suffers from NPD), EVERY SINGLE ONE OF THEM having been in the news and/or in a court for random/assorted “unpleasant” behaviors (privacy abuses, eavesdropping, monopoly abuse, sexual or professional harassment, you just name it…), and EVERY SINGLE ONE OF THEM growing user bases that far exceeds the total population of multiple countries combined.

News Roundup

The HamBSD project aims to bring amateur packet radio to OpenBSD

The HamBSD project aims to bring amateur packet radio to OpenBSD, including support for TCP/IP over AX.25 and APRS tracking/digipeating in the base system.

HamBSD will not provide a full AX.25 stack but instead only implement support for UI frames. There will be a focus on simplicity, security and readable code.

The amateur radio community needs a reliable platform for packet radio for use in both leisure and emergency scenarios. It should be expected that the system is stable and resilient (but as yet it is neither).

DragonFlyBSD's HAMMER2 Gets Basic FSCK Support

HAMMER2 is Copy on Write, meaning changes are made to copies of existing data. This means operations are generally atomic and can survive a power outage, etc. (You should read up on it!) However, there’s now a fsck command, useful if you want a report of data validity rather than any manual repair process.

Add initial fsck support for HAMMER2, although CoW fs doesn't require fsck as a concept. Currently no repairing (no write), just verifying.

Keep this as a separate command for now.
https://i.redd.it/vkdss0mtdpo31.jpg

The return of startx for users

Add modesetting driver as a fall-back when appropriate such that we can use it when running without root privileges which prevents us from scanning the PCI bus.

This makes startx(1)/xinit(1) work again on modern systems with inteldrm(4), radeondrm(4) and amdgpu(4). In some cases this will result in using a different driver than with xenodm(4) which may expose issues (e.g. when we prefer the intel Xorg driver) or loss of acceleration (e.g. older cards supported by radeondrm(4)).

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Your browser does not support the HTML5 video tag.

Sep 26 2019

52mins

Play

316: git commit FreeBSD

Podcast cover
Read more

NetBSD LLVM sanitizers and GDB regression test suite, Ada—The Language of Cost Savings, Homura - a Windows Games Launcher for FreeBSD, FreeBSD core team appoints a WG to explore transition to Git, OpenBSD 6.6 Beta tagged, Project Trident 12-U5 update now available, and more.

Headlines

LLVM santizers and GDB regression test suite.

As NetBSD-9 is branched, I have been asked to finish the LLVM sanitizer integration. This work is now accomplished and with MKLLVM=yes build option (by default off), the distribution will be populated with LLVM files for ASan, TSan, MSan, UBSan, libFuzzer, SafeStack and XRay.

I have also transplanted basesystem GDB patched to my GDB repository and managed to run the GDB regression test-suite.

  • NetBSD distribution changes

I have enhanced and imported my local MKSANITIZER code that makes whole distribution sanitization possible. Few real bugs were fixed and a number of patches were newly written to reflect the current NetBSD sources state. I have also merged another chunk of the fruits of the GSoC-2018 project with fuzzing the userland (by plusun@).

  • The following changes were committed to the sources:
    • ab7de18d0283 Cherry-pick upstream compiler-rt patches for LLVM sanitizers
    • 966c62a34e30 Add LLVM sanitizers in the MKLLVM=yes build
    • 8367b667adb9 telnetd: Stop defining the same variables concurrently in bss and data
    • fe72740f64bf fsck: Stop defining the same variable concurrently in bss and data
    • 40e89e890d66 Fix build of t_ubsan/t_ubsanxx under MKSANITIZER
    • b71326fd7b67 Avoid symbol clashes in tests/usr.bin/id under MKSANITIZER
    • c581f2e39fa5 Avoid symbol clashes in fs/nfs/nfsservice under MKSANITIZER
    • 030a4686a3c6 Avoid symbol clashes in bin/df under MKSANITIZER
    • fd9679f6e8b1 Avoid symbol clashes in usr.sbin/ypserv/ypserv under MKSANITIZER
    • 5df2d7939ce3 Stop defining _rpcsvcdirty in bss and data
    • 5fafbe8b8f64 Add missing extern declaration of ib_mach_emips in installboot
    • d134584be69a Add SANITIZER_RENAME_CLASSES in bsd.prog.mk
    • 2d00d9b08eae Adapt tests/kernel/t_subr_prf for MKSANITIZER
    • ce54363fe452 Ship with sanitizer/lsan_interface.h for GCC 7
    • 7bd5ee95e9a0 Ship with sanitizer/lsan_interface.h for LLVM 7
    • d8671fba7a78 Set NODEBUG for LLVM sanitizers
    • 242cd44890a2 Add PAXCTL_FLAG rules for MKSANITIZER
    • 5e80ab99d9ce Avoid symbol clashes in test/rump/modautoload/t_modautoload with sanitizers
    • e7ce7ecd9c2a sysctl: Add indirection of symbols to remove clash with sanitizers
    • 231aea846aba traceroute: Add indirection of symbol to remove clash with sanitizers
    • 8d85053f487c sockstat: Add indirection of symbols to remove clash with sanitizers
    • 81b333ab151a netstat: Add indirection of symbols to remove clash with sanitizers
    • a472baefefe8 Correct the memset(3)'s third argument in i386 biosdisk.c
    • 7e4e92115bc3 Add ATF c and c++ tests for TSan, MSan, libFuzzer
    • 921ddc9bc97c Set NOSANITIZER in i386 ramdisk image
    • 64361771c78d Enhance MKSANITIZER support
    • 3b5608f80a2b Define target_not_supported_body() in TSan, MSan and libFuzzer tests
    • c27f4619d513 Avoids signedness bit shift in db_get_value()
    • 680c5b3cc24f Fix LLVM sanitizer build by GCC (HAVE_LLVM=no)
    • 4ecfbbba2f2a Rework the LLVM compiler_rt build rules
    • 748813da5547 Correct the build rules of LLVM sanitizers
    • 20e223156dee Enhance the support of LLVM sanitizers
    • 0bb38eb2f20d Register syms.extra in LLVM sanitizer .syms files
    • Almost all of the mentioned commits were backported to NetBSD-9 and will land 9.0.

Homura - a Windows Games Launcher for FreeBSD

Inspired by lutris (a Linux gaming platform), we would like to provide a game launcher to play windows games on FreeBSD.

  • Makes it easier to run games on FreeBSD, by providing the tweaks and dependencies for you
  • Dependencies
    • curl
    • bash
    • p7zip
    • zenity
    • webfonts
    • alsa-utils (Optional)
    • winetricks
    • vulkan-tools
    • mesa-demos
    • i386-wine-devel on amd64 or wine-devel on i386

News Roundup

Ada—The Language of Cost Savings?

Many myths surround the Ada programming language, but it continues to be used and evolve at the same time. And while the increased adoption of Ada and SPARK, its provable subset, is slow, it’s noticeable. Ada already addresses more of the features found in found in heavily used embedded languages like C+ and C#. It also tackles problems addressed by upcoming languages like Rust.

Chris concludes, “Development technologies have a profound impact on one of the largest and most variable costs associated with embedded-system engineering—labor. At a time when on-time system deployment can not only impact customer satisfaction, but access to services revenue streams, engineering team efficiency is at a premium. Our research showed that programming language choices can have significant influence in this area, leading to shorter projects, better schedules and, ultimately, lower development costs. While a variety of factors can influence and dictate language choice, our research showed that Ada’s evolution has made it an increasingly compelling option for engineering organizations, providing both technically and financially sound solution.”

In general, Ada already makes embedded “programming in the large” much easier by handling issues that aren’t even addressed in other languages. Though these features are often provided by third-party software, it results in inconsistent practices among developers. Ada also supports the gamut of embedded platforms from systems like Arm’s Cortex-M through supercomputers. Learning Ada isn’t as hard as one might think and the benefits can be significant.

FreeBSD core team appoints a WG to explore transitioning from Subversion to Git.

  • The FreeBSD Core Team is the governing body of FreeBSD.

Core approved source commit bits for Doug Moore (dougm), Chuck Silvers (chs), Brandon Bergren (bdragon), and a vendor commit bit for Scott Phillips (scottph).

The annual developer survey closed on 2019-04-02. Of the 397 developers, 243 took the survey with an average completion time of 12 minutes. The public survey closed on 2019-05-13. It was taken by 3637 users and had a 79% completion rate. A presentation of the survey results took place at BSDCan 2019.

The core team voted to appoint a working group to explore transitioning our source code 'source of truth' from Subversion to Git. Core asked Ed Maste to chair the group as Ed has been researching this topic for some time. For example, Ed gave a MeetBSD 2018 talk on the topic.

There is a variety of viewpoints within core regarding where and how to host a Git repository, however core feels that Git is the prudent path forward.

OpenBSD 6.6 Beta tagged

CVSROOT: /cvs Module name: src Changes by: deraadt@cvs.openbsd.org 2019/08/09 21:56:02 Modified files: etc/root : root.mail share/mk : sys.mk sys/arch/macppc/stand/tbxidata: bsd.tbxi sys/conf : newvers.sh sys/sys : param.h usr.bin/signify: signify.1 Log message: move to 6.6-beta

Preliminary release notes

Improved hardware support, including:

  • clang(1) is now provided on powerpc.
  • IEEE 802.11 wireless stack improvements:
  • Generic network stack improvements:
  • Installer improvements:
  • Security improvements:
  • + Routing daemons and other userland network improvements
  • + The ntpd(8) daemon now gets and sets the clock in a secure way when booting even when a battery-backed clock is absent.
  • + bgdp(8) improvements
  • + Assorted improvements:
  • + The filesystem buffer cache now more aggressively uses memory outside the DMA region, to improve cache performance on amd64 machines.
  • The BER API previously internal to ldap(1), ldapd(8), ypldap(8), and snmpd(8) has been moved into libutil. See ber_read_elements(3).
  • Support for specifying boot device in vm.conf(5).
  • OpenSMTPD 6.6.0
  • LibreSSL 3.0.X
  • API and Documentation Enhancements
  • Completed the port of RSA_METHOD accessors from the OpenSSL 1.1 API.
  • Documented undescribed options and removed unfunctional options description in openssl(1) manual.
  • OpenSSH 8.0

Project Trident 12-U5 update now available

This is the fifth general package update to the STABLE release repository based upon TrueOS 12-Stable.

  • Package changes from Stable 12-U4
  • Package Summary

    • New Packages: 20
    • Deleted Packages: 24
    • Updated Packages: 279
  • New Packages (20)

    • artemis (biology/artemis) : 17.0.1.11
    • catesc (games/catesc) : 0.6
    • dmlc-core (devel/dmlc-core) : 0.3.105
    • go-wtf (sysutils/go-wtf) : 0.20.0_1
    • instead (games/instead) : 3.3.0_1
    • lidarr (net-p2p/lidarr) : 0.6.2.883
    • minerbold (games/minerbold) : 1.4
    • onnx (math/onnx) : 1.5.0
    • openzwave-devel (comms/openzwave-devel) : 1.6.897
    • polkit-qt-1 (sysutils/polkit-qt) : 0.113.0_8
    • py36-traitsui (graphics/py-traitsui) : 6.1.2
    • rubygem-aws-sigv2 (devel/rubygem-aws-sigv2) : 1.0.1
    • rubygem-default_value_for32 (devel/rubygem-default_value_for32) : 3.2.0
    • rubygem-ffi110 (devel/rubygem-ffi110) : 1.10.0
    • rubygem-zeitwerk (devel/rubygem-zeitwerk) : 2.1.9
    • sems (net/sems) : 1.7.0.g20190822
    • skypat (devel/skypat) : 3.1.1
    • tvm (math/tvm) : 0.4.1440
    • vavoom (games/vavoom) : 1.33_15
    • vavoom-extras (games/vavoom-extras) : 1.30_4
  • Deleted Packages (24)

    • geeqie (graphics/geeqie) : Unknown reason
    • iriverter (multimedia/iriverter) : Unknown reason
    • kde5 (x11/kde5) : Unknown reason
    • kicad-doc (cad/kicad-doc) : Unknown reason
    • os-nozfs-buildworld (os/buildworld) : Unknown reason
    • os-nozfs-userland (os/userland) : Unknown reason
    • os-nozfs-userland-base (os/userland-base) : Unknown reason
    • os-nozfs-userland-base-bootstrap (os/userland-base-bootstrap) : Unknown reason
    • os-nozfs-userland-bin (os/userland-bin) : Unknown reason
    • os-nozfs-userland-boot (os/userland-boot) : Unknown reason
    • os-nozfs-userland-conf (os/userland-conf) : Unknown reason
    • os-nozfs-userland-debug (os/userland-debug) : Unknown reason
    • os-nozfs-userland-devtools (os/userland-devtools) : Unknown reason
    • os-nozfs-userland-docs (os/userland-docs) : Unknown reason
    • os-nozfs-userland-lib (os/userland-lib) : Unknown reason
    • os-nozfs-userland-lib32 (os/userland-lib32) : Unknown reason
    • os-nozfs-userland-lib32-development (os/userland-lib32-development) : Unknown reason
    • os-nozfs-userland-rescue (os/userland-rescue) : Unknown reason
    • os-nozfs-userland-sbin (os/userland-sbin) : Unknown reason
    • os-nozfs-userland-tests (os/userland-tests) : Unknown reason
    • photoprint (print/photoprint) : Unknown reason
    • plasma5-plasma (x11/plasma5-plasma) : Unknown reason
    • polkit-qt5 (sysutils/polkit-qt) : Unknown reason
    • secpanel (security/secpanel) : Unknown reason

Beastie Bits

Feedback/Questions

[CHVT feedback]
DJ - Feedback
Ben - chvt
Harri - Marc's chvt question

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Your browser does not support the HTML5 video tag.

Sep 19 2019

1hr 5mins

Play

315: Recapping vBSDcon 2019

Podcast cover
Read more

vBSDcon 2019 recap, Unix at 50, OpenBSD on fan-less Tuxedo InfinityBook, humungus - an hg server, how to configure a network dump in FreeBSD, and more.

Headlines

vBSDcon Recap

Allan and Benedict attended vBSDcon 2019, which ended last week.

It was held again at the Hyatt Regency Reston and the main conference was organized by Dan Langille of BSDCan fame.The two day conference was preceded by a one day FreeBSD hackathon, where FreeBSD developers had the chance to work on patches and PRs. In the evening, a reception was held to welcome attendees and give them a chance to chat and get to know each other over food and drinks.

The first day of the conference was opened with a Keynote by Paul Vixie about DNS over HTTPS (DoH). He explained how we got to the current state and what challenges (technical and social) this entails.

  • If you missed this talk and are dying to see it, it will also be presented at EuroBSDCon next week

John Baldwin followed up by giving an overview of the work on “In-Kernel TLS Framing and Encryption for FreeBSD” abstract and the recent commit we covered in episode 313.

Meanwhile, Brian Callahan was giving a separate session in another room about “Learning to (Open)BSD through its porting system: an attendee-driven educational session” where people had the chance to learn about how to create ports for the BSDs.

David Fullard’s talk about “Transitioning from FreeNAS to FreeBSD” was his first talk at a BSD conference and described how he built his own home NAS setup trying to replicate FreeNAS’ functionality on FreeBSD, and why he transitioned from using an appliance to using vanilla FreeBSD.

Shawn Webb followed with his overview talk about the “State of the Hardened Union”.

Benedict’s talk about “Replacing an Oracle Server with FreeBSD, OpenZFS, and PostgreSQL” was well received as people are interested in how we liberated ourselves from the clutches of Oracle without compromising functionality.

Entertaining and educational at the same time, Michael W. Lucas talk about “Twenty Years in Jail: FreeBSD Jails, Then and Now” closed the first day. Lucas also had a table in the hallway with his various tech and non-tech books for sale.

People formed small groups and went into town for dinner. Some returned later that night to some work in the hacker lounge or talk amongst fellow BSD enthusiasts.

Colin Percival was the keynote speaker for the second day and had an in-depth look at “23 years of software side channel attacks”.

Allan reprised his “ELI5: ZFS Caching” talk explaining how the ZFS adaptive replacement cache (ARC) work and how it can be tuned for various workloads.

“By the numbers: ZFS Performance Results from Six Operating Systems and Their Derivatives” by Michael Dexter followed with his approach to benchmarking OpenZFS on various platforms.

Conor Beh was also a new speaker to vBSDcon. His talk was about “FreeBSD at Work: Building Network and Storage Infrastructure with pfSense and FreeNAS”.

Two OpenBSD talks closed the talk session: Kurt Mosiejczuk with “Care and Feeding of OpenBSD Porters” and Aaron Poffenberger with “Road Warrior Disaster Recovery: Secure, Synchronized, and Backed-up”.

A dinner and reception was enjoyed by the attendees and gave more time to discuss the talks given and other things until late at night.

We want to thank the vBSDcon organizers and especially Dan Langille for running such a great conference. We are grateful to Verisign as the main sponsor and The FreeBSD Foundation for sponsoring the tote bags. Thanks to all the speakers and attendees!

humungus - an hg server

  • Features

    • View changes, files, changesets, etc. Some syntax highlighting.
    • Read only.
    • Serves multiple repositories.
    • Allows cloning via the obvious URL. Supports go get.
    • Serves files for downloads.
    • Online documentation via mandoc.
    • Terminal based admin interface.

News Roundup

OpenBSD on fan-less Tuxedo InfinityBook 14″ v2.

The InfinityBook 14” v2 is a fanless 14” notebook. It is an excellent choice for running OpenBSD - but order it with the supported wireless card (see below.).

I’ve set it up in a dual-boot configuration so that I can switch between Linux and OpenBSD - mainly to spot differences in the drivers. TUXEDO allows a variety of configurations through their webshop.

The dual boot setup with grub2 and EFI boot will be covered in a separate blogpost. My tests were done with OpenBSD-current - which is as of writing flagged as 6.6-beta.

  • See Article for breakdown of CPU, Wireless, Video, Webcam, Audio, ACPI, Battery, Touchpad, and MicroSD Card Reader

Unix at 50: How the OS that powered smartphones started from failure

Maybe its pervasiveness has long obscured its origins. But Unix, the operating system that in one derivative or another powers nearly all smartphones sold worldwide, was born 50 years ago from the failure of an ambitious project that involved titans like Bell Labs, GE, and MIT. Largely the brainchild of a few programmers at Bell Labs, the unlikely story of Unix begins with a meeting on the top floor of an otherwise unremarkable annex at the sprawling Bell Labs complex in Murray Hill, New Jersey.

It was a bright, cold Monday, the last day of March 1969, and the computer sciences department was hosting distinguished guests: Bill Baker, a Bell Labs vice president, and Ed David, the director of research. Baker was about to pull the plug on Multics (a condensed form of MULTiplexed Information and Computing Service), a software project that the computer sciences department had been working on for four years. Multics was two years overdue, way over budget, and functional only in the loosest possible understanding of the term.

Trying to put the best spin possible on what was clearly an abject failure, Baker gave a speech in which he claimed that Bell Labs had accomplished everything it was trying to accomplish in Multics and that they no longer needed to work on the project. As Berk Tague, a staffer present at the meeting, later told Princeton University, “Like Vietnam, he declared victory and got out of Multics.”

Within the department, this announcement was hardly unexpected. The programmers were acutely aware of the various issues with both the scope of the project and the computer they had been asked to build it for.

Still, it was something to work on, and as long as Bell Labs was working on Multics, they would also have a $7 million mainframe computer to play around with in their spare time. Dennis Ritchie, one of the programmers working on Multics, later said they all felt some stake in the success of the project, even though they knew the odds of that success were exceedingly remote.

Cancellation of Multics meant the end of the only project that the programmers in the Computer science department had to work on—and it also meant the loss of the only computer in the Computer science department. After the GE 645 mainframe was taken apart and hauled off, the computer science department’s resources were reduced to little more than office supplies and a few terminals.

  • Some of Allan’s favourite excerpts:

In the early '60s, Bill Ninke, a researcher in acoustics, had demonstrated a rudimentary graphical user interface with a DEC PDP-7 minicomputer. Acoustics still had that computer, but they weren’t using it and had stuck it somewhere out of the way up on the sixth floor.

And so Thompson, an indefatigable explorer of the labs’ nooks and crannies, finally found that PDP-7 shortly after Davis and Baker cancelled Multics.

With the rest of the team’s help, Thompson bundled up the various pieces of the PDP-7—a machine about the size of a refrigerator, not counting the terminal—moved it into a closet assigned to the acoustics department, and got it up and running. One way or another, they convinced acoustics to provide space for the computer and also to pay for the not infrequent repairs to it out of that department’s budget.

McIlroy’s programmers suddenly had a computer, kind of. So during the summer of 1969, Thompson, Ritchie, and Canaday hashed out the basics of a file manager that would run on the PDP-7. This was no simple task. Batch computing—running programs one after the other—rarely required that a computer be able to permanently store information, and many mainframes did not have any permanent storage device (whether a tape or a hard disk) attached to them. But the time-sharing environment that these programmers had fallen in love with required attached storage. And with multiple users connected to the same computer at the same time, the file manager had to be written well enough to keep one user’s files from being written over another user’s. When a file was read, the output from that file had to be sent to the user that was opening it.

It was a challenge that McIlroy’s team was willing to accept. They had seen the future of computing and wanted to explore it. They knew that Multics was a dead-end, but they had discovered the possibilities opened up by shared development, shared access, and real-time computing. Twenty years later, Ritchie characterized it for Princeton as such: “What we wanted to preserve was not just a good environment in which to do programming, but a system around which a fellowship could form.”

Eventually when they had the file management system more or less fleshed out conceptually, it came time to actually write the code. The trio—all of whom had terrible handwriting—decided to use the Labs’ dictating service. One of them called up a lab extension and dictated the entire code base into a tape recorder. And thus, some unidentified clerical worker or workers soon had the unenviable task of trying to convert that into a typewritten document.

Of course, it was done imperfectly. Among various errors, “inode” came back as “eye node,” but the output was still viewed as a decided improvement over their assorted scribbles.

In August 1969, Thompson’s wife and son went on a three-week vacation to see her family out in Berkeley, and Thompson decided to spend that time writing an assembler, a file editor, and a kernel to manage the PDP-7 processor. This would turn the group’s file manager into a full-fledged operating system. He generously allocated himself one week for each task.

Thompson finished his tasks more or less on schedule. And by September, the computer science department at Bell Labs had an operating system running on a PDP-7—and it wasn’t Multics.

By the summer of 1970, the team had attached a tape drive to the PDP-7, and their blossoming OS also had a growing selection of tools for programmers (several of which persist down to this day). But despite the successes, Thompson, Canaday, and Ritchie were still being rebuffed by labs management in their efforts to get a brand-new computer.

It wasn’t until late 1971 that the computer science department got a truly modern computer. The Unix team had developed several tools designed to automatically format text files for printing over the past year or so. They had done so to simplify the production of documentation for their pet project, but their tools had escaped and were being used by several researchers elsewhere on the top floor. At the same time, the legal department was prepared to spend a fortune on a mainframe program called “AstroText.” Catching wind of this, the Unix crew realized that they could, with only a little effort, upgrade the tools they had written for their own use into something that the legal department could use to prepare patent applications.

The computer science department pitched lab management on the purchase of a DEC PDP-11 for document production purposes, and Max Mathews offered to pay for the machine out of the acoustics department budget. Finally, management gave in and purchased a computer for the Unix team to play with. Eventually, word leaked out about this operating system, and businesses and institutions with PDP-11s began contacting Bell Labs about their new operating system. The Labs made it available for free—requesting only the cost of postage and media from anyone who wanted a copy.

The rest has quite literally made tech history.

  • See the link for the rest of the article

How to configure a network dump in FreeBSD?

A network dump might be very useful for collecting kernel crash dumps from embedded machines and machines with a larger amount of RAM then available swap partition size. Besides net dumps we can also try to compress the core dump. However, often this may still not be enough swap to keep whole core dump. In such situation using network dump is a convenient and reliable way for collecting kernel dump.

So, first, let’s talk a little bit about history. The first implementation of the network dumps was implemented around 2000 for the FreeBSD 4.x as a kernel module. The code was implemented in 2010 with the intention of being part of FreeBSD 9.0. However, the code never landed in FreeBSD. Finally, in 2018 with the commit r333283 by Mark Johnston the netdump client code landed in the FreeBSD. Subsequently, many other commitments were then implemented to add support for the different drivers (for example r333289). The first official release of FreeBSD, which support netdump is FreeBSD 12.0.

Now, let’s get back to the main topic. How to configure the network dump? Two machines are needed. One machine is to collect core dump, let’s call it server. We will use the second one to send us the core dump - the client.

  • See the link for the rest of the article

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

Sep 12 2019

1hr 16mins

Play

314: Swap that Space

Podcast cover
Read more

Unix virtual memory when you have no swap space, Dsynth details on Dragonfly, Instant Workstation on FreeBSD, new servers new tech, Experimenting with streaming setups on NetBSD, NetBSD’s progress towards Steam support thanks to GSoC, and more.

Headlines

What has to happen with Unix virtual memory when you have no swap space

Recently, Artem S. Tashkinov wrote on the Linux kernel mailing list about a Linux problem under memory pressure (via, and threaded here). The specific reproduction instructions involved having low RAM, turning off swap space, and then putting the system under load, and when that happened (emphasis mine):

Once you hit a situation when opening a new tab requires more RAM than is currently available, the system will stall hard. You will barely be able to move the mouse pointer. Your disk LED will be flashing incessantly (I'm not entirely sure why). [...]

I'm afraid I have bad news for the people snickering at Linux here; if you're running without swap space, you can probably get any Unix to behave this way under memory pressure. If you can't on your particular Unix, I'd actually say that your Unix is probably not letting you get full use out of your RAM.

To simplify a bit, we can divide pages of user memory up into anonymous pages and file-backed pages. File-backed pages are what they sound like; they come from some specific file on the filesystem that they can be written out to (if they're dirty) or read back in from. Anonymous pages are not backed by a file, so the only place they can be written out to and read back in from is swap space. Anonymous pages mostly come from dynamic memory allocations and from modifying the program's global variables and data; file backed pages come mostly from mapping files into memory with mmap() and also, crucially, from the code and read-only data of the program.

  • See link for the rest of the article

Dsynth details on Dragonfly

First, history: DragonFly has had binaries of dports available for download for quite some time. These were originally built using poudriere, and then using the synth tool put together by John Marino. Synth worked both to build all software in dports, and as a way to test DragonFly’s SMP capability under extreme load.

Matthew Dillon is working on a new version, called dsynth. It is available now but not yet part of the build. He’s been working quickly on it and there’s plenty more commits than what I have linked here. It’s already led to finding more high-load fixes.

  • dsynth

DSynth is basically synth written in C, from scratch. It is designed to give us a bulk builder in base and be friendly to porting and jails down the line (for now its uses chroot's).

The original synth was written by John R. Marino and its basic flow was used in writing this program, but as it was written in ada no code was directly copied.

  • The intent is to make dsynth compatible with synth's configuration files and directory structure.

  • This is a work in progress and not yet ready for prime-time. Pushing so we can get some more eyeballs. Most of the directives do not yet work (everything, and build works, and 'cleanup' can be used to clean up any dangling mounts).

News Roundup

Instant Workstation

Some considerable time ago I wrote up instructions on how to set up a FreeBSD machine with the latest KDE Plasma Desktop. Those instructions, while fairly short (set up X, install the KDE meta-port, .. and that’s it) are a bit fiddly.

So – prompted slightly by a Twitter exchange recently – I’ve started a mini-sub-project to script the installation of a desktop environment and the bits needed to support it. To give it at least a modicum of UI, dialog(1) is used to ask for an environment to install and a display manager.

The tricky bits – pointed out to me after I started – are hardware support, although a best-effort is better than having nothing, I think.

In any case, in a VBox host it’s now down to running a single script and picking Plasma and SDDM to get a usable system for me. Other combinations have not been tested, nor has system-hardware-setup. I’ll probably maintain it for a while and if I have time and energy it’ll be tried with nVidia (those work quite well on FreeBSD) and AMD (not so much, in my experience) graphics cards when I shuffle some machines around.

New Servers, new Tech

Following up on an earlier post, the new servers for DragonFly are in place. The old 40-core machine used for bulk build, monster, is being retired. The power efficiency of the new machines is startling. Incidentally, this is where donations go – infrastructure.

We have three new servers in the colo now that will be taking most/all bulk package building duties from monster and the two blades (muscles and pkgbox64) that previously did the work. Monster will be retired. The new servers are a dual-socket Xeon (sting) and two 3900X based systems (thor and loki) which all together burn only around half the wattage that monster burned (500W vs 1000W) and 3 times the performance. That's at least a 6:1 improvement in performance efficiency.

With SSD prices down significantly the new machines have all-SSDs. These new machines allow us to build dports binary packages for release, master, and staged at the same time and reduces the full-on bulk build times for getting all three done down from 2 weeks to 2 days. It will allow us to more promptly synchronize updates to ports with dports and get binary packages up sooner.

Monster, our venerable 48-core quad-socket opteron is being retired. This was a wonderful dev machine for working on DragonFly's SMP algorithms over the last 6+ years precisely because its inter-core and inter-socket latencies were quite high. If a SMP algorithm wasn't spot-on, you could feel it. Over the years DragonFly's performance on monster in doing things like bulk builds increased radically as the SMP algorithms got better and the cores became more and more localized. This kept monster relevant far longer than I thought it would be.

But we are at a point now where improvements in efficiency are just too good to ignore. Monster's quad-socket opteron (4 x 12 core 6168's) pulls 1000W under full load while a single Ryzen 3900X (12 core / 24 thread) in a server configuration pulls only 150W, and is slightly faster on the same workload to boot.

I would like to thank everyone's generous donations over the last few years! We burned a few thousand on the new machines (as well as the major SSD upgrades we did to the blades) and made very good use of the money, particularly this year as prices for all major components (RAM, SSDs, CPUs, Mobos, etc) have dropped significantly.

Experimenting with streaming setups on NetBSD

Ever since OBS was successfully ported to NetBSD, I’ve been trying it out, seeing what works and what doesn’t. I’ve only just gotten started, and there’ll definitely be a lot of tweaking going forward.

Capturing a specific application’s windows seems to work okay. Capturing an entire display works, too. I actually haven’t tried streaming to Twitch or YouTube yet, but in a previous experiment a few weeks ago, I was able to run a FFmpeg command line and that could stream to Twitch mostly OK.

My laptop combined with my external monitor allows me to have a dual-monitor setup wherein the smaller laptop screen can be my “broadcasting station” while the bigger screen is where all the action takes place. I can make OBS visible on all Xfce workspaces, but keep it tucked away on that display only. Altogether, the setup should let me use the big screen for the fun stuff but I can still monitor everything in the small screen.

NetBSD Made Progress Thanks To GSoC In Its March Towards Steam Support

Ultimately the goal is to get Valve's Steam client running on NetBSD using their Linux compatibility layer while the focus the past few months with Google Summer of Code 2019 were supporting the necessary DRM ioctls for allowing Linux software running on NetBSD to be able to tap accelerated graphics support.

Student developer Surya P spent the summer working on compat_netbsd32 DRM interfaces to allow Direct Rendering Manager using applications running under their Linux compatibility layer.

These interfaces have been tested and working as well as updating the "suse131" packages in NetBSD to make use of those interfaces. So the necessary interfaces are now in place for Linux software running on NetBSD to be able to use accelerated graphics though Steam itself isn't yet running on NetBSD with this layer.

Those curious about this DRM ioctl GSoC project can learn more from the NetBSD blog. NetBSD has also been seeing work this summer on Wayland support and better Wine support to ultimately make this BSD a better desktop operating system and potentially a comparable gaming platform to Linux.

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

Sep 05 2019

48mins

Play

iTunes Ratings

61 Ratings
Average Ratings
56
3
1
1
0

Great for keeping up wth BSDs and Unix

By brysonholland - Mar 19 2019
Read more
They don&#39;t just cover the BSDs, but the wider Unix world as well. Great topics, technical analysis, and interviews. One of the best Unix/BSD podcasts currently out there.

Best beginner-accessible BSD resource anywhere!

By ProCoreyJ - Dec 12 2016
Read more
This is the best source ofBSD info available to beginners but also doesn&#39;t hesitate to dive into the interesting details holding it all together. I have found this podcast to be an essential resource for getting and staying current with tools, techniques, and community to make BSD work for me. Thanks for putting this on! Additionally, I think this podcast is probably the best marketing arm for the community. Keep it available!