Cover image of BSD Now
(65)

Rank #113 in Tech News category

Education
How To
News
Tech News

BSD Now

Updated 2 months ago

Rank #113 in Tech News category

Education
How To
News
Tech News
Read more

Created by three guys who love BSD, we cover the latest news and have an extensive series of tutorials, as well as interviews with various people from all areas of the BSD community. It also serves as a platform for support and questions. We love and advocate FreeBSD, OpenBSD, NetBSD, DragonFlyBSD and TrueOS. Our show aims to be helpful and informative for new users that want to learn about them, but still be entertaining for the people who are already pros.The show airs on Wednesdays at 2:00PM (US Eastern time) and the edited version is usually up the following day.

Read more

Created by three guys who love BSD, we cover the latest news and have an extensive series of tutorials, as well as interviews with various people from all areas of the BSD community. It also serves as a platform for support and questions. We love and advocate FreeBSD, OpenBSD, NetBSD, DragonFlyBSD and TrueOS. Our show aims to be helpful and informative for new users that want to learn about them, but still be entertaining for the people who are already pros.The show airs on Wednesdays at 2:00PM (US Eastern time) and the edited version is usually up the following day.

iTunes Ratings

65 Ratings
Average Ratings
59
4
1
1
0

Great for keeping up wth BSDs and Unix

By brysonholland - Mar 19 2019
Read more
They don't just cover the BSDs, but the wider Unix world as well. Great topics, technical analysis, and interviews. One of the best Unix/BSD podcasts currently out there.

Best beginner-accessible BSD resource anywhere!

By ProCoreyJ - Dec 12 2016
Read more
This is the best source ofBSD info available to beginners but also doesn't hesitate to dive into the interesting details holding it all together. I have found this podcast to be an essential resource for getting and staying current with tools, techniques, and community to make BSD work for me. Thanks for putting this on! Additionally, I think this podcast is probably the best marketing arm for the community. Keep it available!

iTunes Ratings

65 Ratings
Average Ratings
59
4
1
1
0

Great for keeping up wth BSDs and Unix

By brysonholland - Mar 19 2019
Read more
They don't just cover the BSDs, but the wider Unix world as well. Great topics, technical analysis, and interviews. One of the best Unix/BSD podcasts currently out there.

Best beginner-accessible BSD resource anywhere!

By ProCoreyJ - Dec 12 2016
Read more
This is the best source ofBSD info available to beginners but also doesn't hesitate to dive into the interesting details holding it all together. I have found this podcast to be an essential resource for getting and staying current with tools, techniques, and community to make BSD work for me. Thanks for putting this on! Additionally, I think this podcast is probably the best marketing arm for the community. Keep it available!
Cover image of BSD Now

BSD Now

Latest release on Aug 06, 2020

Read more

Created by three guys who love BSD, we cover the latest news and have an extensive series of tutorials, as well as interviews with various people from all areas of the BSD community. It also serves as a platform for support and questions. We love and advocate FreeBSD, OpenBSD, NetBSD, DragonFlyBSD and TrueOS. Our show aims to be helpful and informative for new users that want to learn about them, but still be entertaining for the people who are already pros.The show airs on Wednesdays at 2:00PM (US Eastern time) and the edited version is usually up the following day.

Rank #1: Episode 250: BSDCan 2018 Recap | BSD Now 250

Podcast cover
Read more

TrueOS becoming a downstream fork with Trident, our BSDCan 2018 recap, HardenedBSD Foundation founding efforts, VPN with OpenIKED on OpenBSD, FreeBSD on a System76 Galago Pro, and hardware accelerated crypto on Octeons.

##Headlines##
###TrueOS to Focus on Core Operating System

The TrueOS Project has some big plans in the works, and we want to take a minute and share them with you. Many have come to know TrueOS as the “graphical FreeBSD” that makes things easy for newcomers to the BSDs. Today we’re announcing that TrueOS is shifting our focus a bit to become a cutting-edge operating system that keeps all of the stability that you know and love from ZFS (OpenZFS) and FreeBSD, and adds additional features to create a fresh, innovative operating system. Our goal is to create a core-centric operating system that is modular, functional, and perfect for do-it-yourselfers and advanced users alike.

TrueOS will become a downstream fork that will build on FreeBSD by integrating new software technologies like OpenRC and LibreSSL. Work has already begun which allows TrueOS to be used as a base platform for other projects, including JSON-based manifests, integrated Poudriere / pkg tools and much more. We’re planning on a six month release cycle to keep development moving and fresh, allowing us to bring you hot new features to ZFS, bhyve and related tools in a timely manner. This makes TrueOS the perfect fit to serve as the basis for building other distributions.

Some of you are probably asking yourselves “But what if I want to have a graphical desktop?” Don’t worry! We’re making sure that everyone who knows and loves the legacy desktop version of TrueOS will be able to continue using a FreeBSD-based, graphical operating system in the future. For instance, if you want to add KDE, just use sudo pkg install kde and voila! You have your new shiny desktop. Easy right? This allows us to get back to our roots of being a desktop agnostic operating system. If you want to add a new desktop environment, you get to pick the one that best suits your use.

We know that some of you will still be looking for an out-of-the-box solution similar to legacy PC-BSD and TrueOS. We’re happy to announce that Project Trident will take over graphical FreeBSD development going forward. Not much is going to change in that regard other than a new name! You’ll still have Lumina Desktop as a lightweight and feature-rich desktop environment and tons of utilities from the legacy TrueOS toolchain like sysadm and AppCafe. There will be migration paths available for those that would like to move to other FreeBSD-based distributions like Project Trident or GhostBSD.

We look forward to this new chapter for TrueOS and hope you will give the new edition a spin! Tell us what you think about the new changes by leaving us a comment. Don’t forget you can ask us questions on our Twitter and be a part of our community by joining the new TrueOS Forums when they go live in about a week. Thanks for being a loyal fan of TrueOS.

###Project Trident FAQ

  • Q: Why did you pick the name “Project Trident”?

A: We were looking for a name that was unique, yet would still relate to the BSD community. Since Beastie (the FreeBSD mascot) is always pictured with a trident, it felt like that would be a great name.

  • Q: Where can users go for technical support?

A: At the moment, Project Trident will continue sharing the TrueOS community forums and Telegram channels. We are currently evaluating dedicated options for support channels in the future.

  • Q: Can I help contribute to the project?

A: We are always looking for developers who want to join the project. If you’re not a developer you can still help, as a community project we will be more reliant on contributions from the community in the form of how-to guides and other user-centric documentation and support systems.

  • Q: How is the project supported financially?

A: Project Trident is sponsored by the community, from both individuals and corporations. iXsystems has stepped up as the first enterprise-level sponsor of the project, and has been instrumental in getting Project Trident up and running. Please visit the Sponsors page to see all the current sponsors.

  • Q: How can I help support the project financially?

A: Several methods exist, from one time or recurring donations via Paypal to limited time swag t-shirt campaigns during the year. We are also looking into more alternative methods of support, so please visit the Sponsors page to see all the current methods of sponsorship.

  • Q: Will there be any transparency of the financial donations and expenditures?

A: Yes, we will be totally open with how much money comes into the project and what it is spent on. Due to concerns of privacy, we will not identify individuals and their donation amounts unless they specifically request to be identified. We will release a monthly overview in/out ledger, so that community members can see where their money is going.

  • Relationship with TrueOS

  • Project Trident does have very close ties to the TrueOS project, since most of the original Project Trident developers were once part of the TrueOS project before it became a distribution platform. For users of the TrueOS desktop, we have some additional questions and answers below.

  • Q: Do we need to be at a certain TrueOS install level/release to upgrade?

A: As long as you have a TrueOS system which has been updated to at least the 18.03 release you should be able to just perform a system update to be automatically upgraded to Project Trident.

  • Q: Which members moved from TrueOS to Project Trident?

A: Project Trident is being led by prior members of the TrueOS desktop team. Ken and JT (development), Tim (documentation) and Rod (Community/Support). Since Project Trident is a community-first project, we look forward to working with new members of the team.

iXsystems

###BSDCan

  • BSDCan finished Saturday last week
  • It started with the GoatBoF on Tuesday at the Royal Oak Pub, where people had a chance to meet and greet. Benedict could not attend due to an all-day FreeBSD Foundation meeting and and even FreeBSD Journal Editorial Board meeting.
  • The FreeBSD devsummit was held the next two days in parallel to the tutorials. Gordon Tetlow, who organized the devsummit, opened the devsummit. Deb Goodkin from the FreeBSD Foundation gave the first talk with a Foundation update, highlighting current and future efforts. Li-Wen Hsu is now employed by the Foundation to assist in QA work (Jenkins, CI/CD) and Gordon Tetlow has a part-time contract to help secteam as their secretary.
  • Next, the FreeBSD core team (among them Allan and Benedict) gave a talk about what has happened this last term. With a core election currently running, some of these items will carry over to the next core team, but there were also some finished ones like the FCP process and FreeBSD members initiative. People in the audience asked questions on various topics of interest.
  • After the coffee break, the release engineering team gave a talk about their efforts in terms of making releases happen in time and good quality.
  • Benedict had to give his Ansible tutorial in the afternoon, which had roughly 15 people attending. Most of them beginners, we could get some good discussions going and I also learned a few new tricks. The overall feedback was positive and one even asked what I’m going to teach next year.
  • The second day of the FreeBSD devsummit began with Gordon Tetlow giving an insight into the FreeBSD Security team (aka secteam). He gave a overview of secteam members and responsibilities, explaining the process based on a long past advisory. Developers were encouraged to help out secteam. NDAs and proper disclosure of vulnerabilities were also discussed, and the audience had some feedback and questions.
  • When the coffee break was over, the FreeBSD 12.0 planning session happened. A Google doc served as a collaborative way of gathering features and things left to do. People signed up for it or were volunteered. Some features won’t make it into 12.0 as they are not 100% ready for prime time and need a few more rounds of testing and bugfixing. Still, 12.0 will have some compelling features.
  • A 360° group picture was taken after lunch, and then people split up into the working groups for the afternoon or started hacking in the UofO Henderson residence.
  • Benedict and Allan both attended the OpenZFS working group, lead by Matt Ahrens. He presented the completed and outstanding work in FreeBSD, without spoiling too much of the ZFS presentations of various people that happened later at the conference.
  • Benedict joined the boot code session a bit late (hallway track is the reason) when most things seem to have already been discussed.
  • BSDCan 2018 — Ottawa (In Pictures)
  • iXsystems Photos from BSDCan 2018

##News Roundup
###June HardenedBSD Foundation Update

We at HardenedBSD are working towards starting up a 501©(3) not-for-profit organization in the USA. Setting up this organization will allow future donations to be tax deductible. We’ve made progress and would like to share with you the current state of affairs.

We have identified, sent invitations out, and received acceptance letters from six people who will serve on the HardenedBSD Foundation Board of Directors. You can find their bios below. In the latter half of June 2018 or the beginning half of July 2018, we will meet for the first time as a board and formally begin the process of creating the documentation needed to submit to the local, state, and federal tax services.

Here’s a brief introduction to those who will serve on the board:

  • W. Dean Freeman (Advisor): Dean has ten years of professional experience with deploying and security Unix and networking systems, including assessing systems security for government certification and assessing the efficacy of security products. He was introduced to Unix via FreeBSD 2.2.8 on an ISP shell account as a teenager. Formerly, he was the Snort port maintainer for FreeBSD while working in the Sourcefire VRT, and has contributed entropy-related patches to the FreeBSD and HardenedBSD projects – a topic on which he presented at vBSDCon 2017.

  • Ben La Monica (Advisor): Ben is a Senior Technology Manager of Software Engineering at Morningstar, Inc and has been developing software for over 15 years in a variety of languages. He advocates open source software and enjoys tinkering with electronics and home automation.

  • George Saylor (Advisor): George is a Technical Directory at G2, Inc. Mr. Saylor has over 28 years of information systems and security experience in a broad range of disciplines. His core focus areas are automation and standards in the event correlation space as well as penetration and exploitation of computer systems. Mr Saylor was also a co-founder of the OpenSCAP project.

  • Virginia Suydan (Accountant and general administrator): Accountant and general administrator for the HardenedBSD Foundation. She has worked with Shawn Webb for tax and accounting purposes for over six years.

  • Shawn Webb (Director): Co-founder of HardenedBSD and all-around infosec wonk. He has worked and played in the infosec industry, doing both offensive and defensive research, for around fifteen years. He loves open source technologies and likes to frustrate the bad guys.

  • Ben Welch (Advisor): Ben is currently a Security Engineer at G2, Inc. He graduated from Pennsylvania College of Technology with a Bachelors in Information Assurance and Security. Ben likes long walks, beaches, candlelight dinners, and attending various conferences like BSides and ShmooCon.

###Your own VPN with OpenIKED & OpenBSD

Remote connectivity to your home network is something I think a lot of people find desirable. Over the years, I’ve just established an SSH tunnel and use it as a SOCKS proxy, sending my traffic through that. It’s a nice solution for a “poor man’s VPN”, but it can be a bit clunky, and it’s not great having to expose SSH to the world, even if you make sure to lock everything down

I set out the other day to finally do it properly. I’d come across this great post by Gordon Turner: OpenBSD 6.2 VPN Endpoint for iOS and macOS

Whilst it was exactly what I was looking for, it outlined how to set up an L2TP VPN. Really, I wanted IKEv2 for performance and security reasons (I won’t elaborate on this here, if you’re curious about the differences, there’s a lot of content out on the web explaining this).

The client systems I’d be using have native support for IKEv2 (iOS, macOS, other BSD systems). But, I couldn’t find any tutorials in the same vein.

So, let’s get stuck in!

  • A quick note ✍️

This guide will walk through the set up of an IKEv2 VPN using OpenIKED on OpenBSD. It will detail a “road warrior” configuration, and use a PSK (pre-shared-key) for authentication. I’m sure it can be easily adapted to work on any other platforms that OpenIKED is available on, but keep in mind my steps are specifically for OpenBSD.

  • Server Configuration

As with all my home infrastructure, I crafted this set-up declaratively. So, I had the deployment of the VM setup in Terraform (deployed on my private Triton cluster), and wrote the configuration in Ansible, then tied them together using radekg/terraform-provisioner-ansible.

One of the reasons I love Ansible is that its syntax is very simplistic, yet expressive. As such, I feel it fits very well into explaining these steps with snippets of the playbook I wrote. I’ll link the full playbook a bit further down for those interested.

  • See the full article for the information on:
  • sysctl parameters
  • The naughty list (optional)
  • Configure the VPN network interface
  • Configure the firewall
  • Configure the iked service
  • Gateway configuration
  • Client configuration
  • Troubleshooting

DigitalOcean

###FreeBSD on a System76 Galago Pro

Hey all, It’s been a while since I last posted but I thought I would hammer something out here. My most recent purchase was a System76 Galago Pro. I thought, afer playing with POP! OS a bit, is there any reason I couldn’t get BSD on this thing. Turns out the answer is no, no there isnt and it works pretty decently.

To get some accounting stuff out of the way I tested this all on FreeBSD Head and 11.1, and all of it is valid as of May 10, 2018. Head is a fast moving target so some of this is only bound to improve.

  • The hardware

  • Intel Core i5 Gen 8

  • UHD Graphics 620

  • 16 GB DDR4 Ram

  • RTL8411B PCI Express Card Reader

  • RTL8111 Gigabit ethernet controller

  • Intel HD Audio

  • Samsung SSD 960 PRO 512GB NVMe

  • The caveats

There are a few things that I cant seem to make work straight out of the box, and that is the SD Card reader, the backlight, and the audio is a bit finicky. Also the trackpad doesn’t respond to two finger scrolling. The wiki is mostly up to date, there are a few edits that need to be made still but there is a bug where I cant register an account yet so I haven’t made all the changes.

  • Processor

It works like any other Intel processor. Pstates and throttling work.

  • Graphics

The boot menu sets itself to what looks like 1024x768, but works as you expect in a tiny window. The text console does the full 3200x1800 resolution, but the text is ultra tiny. There isnt a font for the console that covers hidpi screens yet. As for X Windows it requres the drm-kmod-next package. Once installed follow the directions from the package and it works with almost no fuss. I have it running on X with full intel acceleration, but it is running at it’s full 3200x1800 resolution, to scale that down just do xrandr --output eDP-1 --scale 0.5x0.5 it will blow it up to roughly 200%. Due to limitations with X windows and hidpi it is harder to get more granular.

  • Intel Wireless 8265

The wireless uses the iwm module, as of right now it does not seem to automagically load right now. Adding iwm_load=“YES” will cause the module to load on boot and kldload iwm

  • Battery

I seem to be getting about 5 hours out of the battery, but everything reports out of the box as expected. I could get more by throttling the CPU down speed wise.

  • Overall impression

It is a pretty decent experience. While not as polished as a Thinkpad there is a lot of potential with a bit of work and polishing. The laptop itself is not bad, the keyboard is responsive. The build quality is pretty solid. My only real complaint is the trackpad is stiff to click and sort of tiny. They seem to be a bit indifferent to non linux OSes running on the gear but that isnt anything new. I wont have any problems using it and is enough that when I work through this laptop, but I’m not sure at this stage if my next machine will be a System76 laptop, but they have impressed me enough to put them in the running when I go to look for my next portable machine but it hasn’t yet replaced the hole left in my heart by lenovo messing with the thinkpad.

###Hardware accelerated AES/HMAC-SHA on octeons

In this commit, visa@ submitted code (disabled for now) to use built-in acceleration on octeon CPUs, much like AESNI for x86s. I decided to test tcpbench(1) and IPsec, before and after updating and enabling the octcrypto(4) driver. I didn't capture detailed perf stats from before the update, I had heard someone say that Edgerouter Lite boxes would only do some 6MBit/s over ipsec, so I set up a really simple ipsec.conf with ike esp from A to B leading to a policy of esp tunnel from A to B spi 0xdeadbeef auth hmac-sha2-256 enc aes going from one ERL to another (I collect octeons, so I have a bunch to test with) and let tcpbench run for a while on it. My numbers hovered around 7Mbit/s, which coincided with what I've heard, and also that most of the CPU gets used while doing it. Then I edited /sys/arch/octeon/conf/GENERIC, removed the # from octcrypto0 at mainbus0 and recompiled. Booted into the new kernel and got a octcrypto0 line in dmesg, and it was time to rock the ipsec tunnel again. The crypto algorithm and HMAC used by default on ipsec coincides nicely with the list of accelerated functions provided by the driver. Before we get to tunnel traffic numbers, just one quick look at what systat pigs says while the ipsec is running at full steam: PID USER NAME CPU 20\ 40\ 60\ 80\ 100\ 58917 root crypto 52.25 ################# 42636 root softnet 42.48 ############## (idle) 29.74 ######### 1059 root tcpbench 24.22 ####### 67777 root crynlk 19.58 ###### So this indicates that the load from doing ipsec and generating the traffic is somewhat nicely evened out over the two cores in the Edgerouter, and there's even some CPU left unused, which means I can actually ssh into it and have it usable. I have had it running for almost 2 days now, moving some 2.1TB over the tunnel. Now for the new and improved performance numbers: 204452123 4740752 37.402 100.00% Conn: 1 Mbps: 37.402 Peak Mbps: 58.870 Avg Mbps: 37.402 204453149 4692968 36.628 100.00% Conn: 1 Mbps: 36.628 Peak Mbps: 58.870 Avg Mbps: 36.628 204454167 5405552 42.480 100.00% Conn: 1 Mbps: 42.480 Peak Mbps: 58.870 Avg Mbps: 42.480 204455188 5202496 40.804 100.00% Conn: 1 Mbps: 40.804 Peak Mbps: 58.870 Avg Mbps: 40.804 204456194 5062208 40.256 100.00% Conn: 1 Mbps: 40.256 Peak Mbps: 58.870 Avg Mbps: 40.256 The tcpbench numbers fluctuate up and down a bit, but the output is nice enough to actually keep tabs on the peak values. Peaking to 58.8MBit/s! Of course, as you can see, the average is lower but nice anyhow. A manyfold increase in performance, which is good enough in itself, but also moves the throughput from a speed that would make a poor but cheap gateway to something actually useful and decent for many home network speeds. Biggest problem after this gets enabled will be that my options to buy cheap used ERLs diminish.

##Beastie Bits

Tarsnap

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Jun 14 2018

1hr 41mins

Play

Rank #2: 347: New Directions

Podcast cover
Read more

Rethinking OpenBSD security, FreeBSD 2020 Q1 status report, the notion of progress and user interfaces, Comments about Thomas E. Dickey on NetBSD curses, making Unix a little more Plan9-like, Not-actually Linux distro review: FreeBSD, and more.

Headlines

Rethinking OpenBSD Security

OpenBSD aims to be a secure operating system. In the past few months there were quite a few security errata, however. That’s not too unusual, but some of the recent ones were a bit special. One might even say bad. The OpenBSD approach to security has a few aspects, two of which might be avoiding errors and minimizing the risk of mistakes. Other people have other ideas about how to build secure systems. I think it’s worth examining whether the OpenBSD approach works, or if this is evidence that it’s doomed to failure.
I picked a few errata, not all of them, that were interesting and happened to suit my narrative.

FreeBSD 2020 Q1 Quarterly report

Welcome, to the quarterly reports, of the future! Well, at least the first quarterly report from 2020. The new timeline, mentioned in the last few reports, still holds, which brings us to this report, which covers the period of January 2020 - March 2020.

News Roundup

The Notion of Progress and User Interfaces

One trait of modern Western culture is the notion of progress. A view claiming, at large, everything is getting better and better.

How should we think about progress? Both in general and regarding technology?

Thomas E. Dickey on NetBSD curses

I was recently pointed at a web page on Thomas E. Dickeys site talking about NetBSD curses. It seems initially that the page was intended to be a pointer to some differences between ncurses and NetBSD curses and does appear to start off in this vein but it seems that the author has lost the plot as the document evolved and the tail end of it seems to be devolving into some sort of slanging match. I don't want to go through Mr. Dickey's document point by point, that would be tedious but I would like to pick out some of the things that I believe to be the most egregious. Please note that even though I am a NetBSD developer, the opinions below are my own and not the NetBSD projects.

Making Unix a little more Plan9-like

I’m not really interested in defending anything. I tried out plan9port and liked it, but I have to live in Unix land. Here’s how I set that up.

A Warning

The suckless community, and some of the plan9 communities, are dominated by jackasses. I hope that’s strong enough wording to impress the severity. Don’t go into IRC for help. Stay off the suckless email list. The software is great, the people who write it are well-spoken and well-reasoned, but for some reason the fandom is horrible to everyone.

Not-actually Linux distro review: FreeBSD 12.1-RELEASE

This month's Linux distro review isn't of a Linux distribution at all—instead, we're taking a look at FreeBSD, the original gangster of free Unix-like operating systems.

The first FreeBSD release was in 1993, but the operating system's roots go further back—considerably further back. FreeBSD started out in 1992 as a patch-release of Bill and Lynne Jolitz's 386BSD—but 386BSD itself came from the original Berkeley Software Distribution (BSD). BSD itself goes back to 1977—for reference, Linus Torvalds was only seven years old then.

Before we get started, I'd like to acknowledge something up front—our distro reviews include the desktop experience, and that is very much not FreeBSD's strength. FreeBSD is far, far better suited to running as a headless server than as a desktop! We're going to get a full desktop running on it anyway, because according to Lee Hutchinson, I hate myself—and also because we can't imagine readers wouldn't care about it.

FreeBSD does not provide a good desktop experience, to say the least. But if you're hankering for a BSD-based desktop, don't worry—we're already planning a followup review of GhostBSD, a desktop-focused BSD distribution.

Beastie Bits

BSDNow is going Independent

  • After being part of Jupiter Broadcasting since we started back in 2013, BSDNow is moving to become independent. We extend a very large thank you to Jupiter Broadcasting and Linux Academy for hosting us for so many years, and allowing us to bring you over 100 episodes without advertisements. LinuxAcademy is now under new leadership, and we understand that cutbacks needed to be made, and that BSD is not their core product. That does not mean your favourite BSD podcast is going away, we will continue and we expect things will not look much different. What does this mean for you, the listener? Not much will change, just make sure your subscription is via the RSS feed at BSDNow.tv rather than one of the Jupiter Broadcasting feeds. We will update you with more news as things settle out.

Feedback/Questions

Your browser does not support the HTML5 video tag.

Apr 23 2020

1hr

Play

Rank #3: Episode 239: The Return To ptrace | BSD Now 239

Podcast cover
Read more

OpenBSD firewalling Windows 10, NetBSD’s return to ptrace, TCP Alternative Backoff, the BSD Poetic license, and AsiaBSDcon 2018 videos available.

RSS Feeds:

MP3 Feed | iTunes Feed | HD Vid Feed | HD Torrent Feed

Become a supporter on Patreon:

- Show Notes: -

Headlines

Preventing Windows 10 and untrusted software from having full access to the internet using OpenBSD

Whilst setting up one of my development laptops to port some software to Windows I noticed Windows 10 doing crazy things like installing or updating apps and games by default after initial setup. The one I noticed in particular was Candy Crush Soda Saga which for those who don't know of it is some cheesy little puzzle game originally for consumer devices. I honestly did not want software like this near to a development machine. It has also been reported that Windows 10 now also updates core system software without notifying the user. Surely this destroys any vaguely deterministic behaviour, in my opinion making Windows 10 by default almost useless for development testbeds.

Deciding instead to start from scratch but this time to set the inbuilt Windows Firewall to be very restrictive and only allow a few select programs to communicate. In this case all I really needed to be online was Firefox, Subversion and Putty. To my amusement (and astonishment) I found out that the Windows firewall could be modified to give access very easily by programs during installation (usually because this task needs to be done with admin privileges). It also seems that Windows store Apps can change the windows firewall settings at any point. One way to get around this issue could be to install a 3rd party firewall that most software will not have knowledge about and thus not attempt to break through. However the only decent firewall I have used was Sygate Pro which unfortunately is no longer supported by recent operating systems. The last supported versions was 2003, XP and 2000. In short, I avoid 3rd party firewalls.

Instead I decided to trap Windows 10 (and all of it's rogue updaters) behind a virtual machine running OpenBSD. This effectively provided me with a full blown firewall appliance. From here I could then allow specific software I trusted through the firewall (via a proxy) in a safe, controlled and deterministic manner. For other interested developers (and security conscious users) and for my own reference, I have listed the steps taken here:

  • 1) First and foremost disable the Windows DHCP service - this is so no IP can be obtained on any interface. This effectively stops any communication with any network on the host system. This can be done by running services.msc with admin privileges and stopping and disabling the service called DHCP Client.

  • 2) Install or enable your favorite virtualization software - I have tested this with both VirtualBox and Hyper-V. Note that on non-server versions of Windows, in order to get Hyper-V working, your processor also needs to support SLAT which is daft so to avoid faffing about, I recommend using VirtualBox to get round this seemingly arbitrary restriction.

  • 3) Install OpenBSD on the VM - Note, if you decide to use Hyper-V, its hardware support isn't 100% perfect to run OpenBSD and you will need to disable a couple of things in the kernel. At the initial boot prompt, run the following commands.


config -e -o /bsd /bsd
disable acpi
disable mpbios

  • 4) Add a host only virtual adapter to the VM - This is the one which we are going to connect through the VM with. Look at the IP that VirtualBox assigns this in network manager on the host machine. Mine was [b]192.168.56.1[/b]. Set up the adapter in the OpenBSD VM to have a static address on the same subnet. For example [b]192.168.56.2[/b]. If you are using Hyper-V and OpenBSD, make sure you add a "Legacy Interface" because no guest additions are available. Then set up a virtual switch which is host only.

  • 5) Add a bridged adapter to the VM - then assign it to whichever interface you wanted to connect to the external network with. Note that if using Wireless, set the bridged adapters MAC address to the same as your physical device or the access point will reject it. This is not needed (or possible) on Hyper-V because the actual device is "shared" rather than bridged so the same MAC address is used. Again, if you use Hyper-V, then add another virtual switch and attach it to your chosen external interface. VMs in Hyper-V "share" an adapter within a virtual switch and there is the option to also disable the hosts ability to use this interface at the same time which is fine for an additional level of security if those pesky rogue apps and updaters can also enable / disable DHCP service one day which wouldn't be too surprising.

  • 6) Connect to your network in the host OS - In case of Wireless, select the correct network from the list and type in a password if needed. Windows will probably say "no internet available", it also does not assign an IP address which is fine.

  • 7) Install the Squid proxy package on the OpenBSD guest and enable the daemon

```

pkg_add squid

echo 'squid_flags=""' >> /etc/rc.conf.local

/etc/rc.d/squid start

```

We will use this service for a limited selection of "safe and trusted" programs to connect to the outside world from within the Windows 10 host. You can also use putty on the host to connect to the VM via SSH and create a SOCKS proxy which software like Firefox can also use to connect externally.

  • 8) Configure the software you want to be able to access the external network with

    • Firefox - go to the connection settings and specify the VMs IP address for the proxy.
    • Subversion - modify the %HOME%\AppData\Roaming\Subversion\servers file and change the HTTP proxy field to the VMs IP. This is important to communicate with GitHub via https:// (Yes, GitHub also supports Subversion). For svn:// addresses you can use Putty to port forward.
    • Chromium/Chrome - unfortunately uses the global Windows proxy settings which defeats much of the purpose of this exercise if we were going to allow all of Windows access to the internet via the proxy. It would become mayhem again. However we can still use Putty to create a SOCKS proxy and then launch the browser with the following flags:


--proxy-server="socks5://:"
--host-resolver-rules="MAP * 0.0.0.0 , EXCLUDE "

  • 9) Congratulations, you are now done - Admittedly this process can be a bit fiddly to set up but it completely prevents Windows 10 from making a complete mess. This solution is probably also useful for those who like privacy or don't like the idea of their software "phoning home". Hope you find this useful and if you have any issues, please feel free to leave questions in the comments.

LLDB restoration and return to ptrace(2)

I've managed to unbreak the LLDB debugger as much as possible with the current kernel and hit problems with ptrace(2) that are causing issues with further work on proper NetBSD support. Meanwhile, I've upstreamed all the planned NetBSD patches to sanitizers and helped other BSDs to gain better or initial support.

  • LLDB

Since the last time I worked on LLDB, we have introduced many changes to the kernel interfaces (most notably related to signals) that apparently fixed some bugs in Go and introduced regressions in ptrace(2). Part of the regressions were noted by the existing ATF tests. However, the breakage was only marked as a new problem to resolve. For completeness, the ptrace(2) code was also cleaned up by Christos Zoulas, and we fixed some bugs with compat32.

I've fixed a crash in *NetBSD::Factory::Launch(), triggered on startup of the lldb-server application.

Here is the commit message:

```
We cannot call process_up->SetState() inside
the NativeProcessNetBSD::Factory::Launch
function because it triggers a NULL pointer
deference.

The generic code for launching a process in:
GDBRemoteCommunicationServerLLGS::LaunchProcess
sets the mdebuggedprocessup pointer after
a successful call to m
processfactory.Launch().
If we attempt to call process
up->SetState()
inside a platform specific Launch function we
end up dereferencing a NULL pointer in
NativeProcessProtocol::GetCurrentThreadID().

Use the proper call processup->SetState(,false)
that sets notify
delegates to false.
```

  • Sanitizers

I suspended development of new features in sanitizers last month, but I was still in the process of upstreaming of local patches. This process was time-consuming as it required rebasing patches, adding dedicated tests, and addressing all other requests and comments from the upstream developers.

I'm not counting hot fixes, as some changes were triggering build or test issues on !NetBSD hosts. Thankfully all these issues were addressed quickly. The final result is a reduction of local delta size of almost 1MB to less than 100KB (1205 lines of diff). The remaining patches are rescheduled for later, mostly because they depend on extra work with cross-OS tests and prior integration of sanitizers with the basesystem distribution. I didn't want to put extra work here in the current state of affairs and, I've registered as a mentor for Google Summer of Code for the NetBSD Foundation and prepared Software Quality improvement tasks in order to outsource part of the labour.

  • Userland changes

I've also improved documentation for some of the features of NetBSD, described in man-pages. These pieces of information were sometimes wrong or incomplete, and this makes covering the NetBSD system with features such as sanitizers harder as there is a mismatch between the actual code and the documented code.

Some pieces of software also require better namespacing support, these days mostly for the POSIX standard. I've fixed few low-hanging fruits there and requested pullups to NetBSD-8(BETA).

I thank the developers for improving the landed code in order to ship the best solutions for users.

  • BSD collaboration in LLVM

A One-man-show in human activity is usually less fun and productive than collaboration in a team. This is also true in software development. Last month I was helping as a reviewer to port LLVM features to FreeBSD and when possible to OpenBSD. This included MSan/FreeBSD, libFuzzer/FreeBSD, XRay/FreeBSD and UBSan/OpenBSD.

I've landed most of the submitted and reviewed code to the mainstream LLVM tree.

Part of the code also verified the correctness of NetBSD routes in the existing porting efforts and showed new options for improvement. This is the reason why I've landed preliminary XRay/NetBSD code and added missing NetBSD bits to ToolChain::getOSLibName(). The latter produced setup issues with the prebuilt LLVM toolchain, as the directory name with compiler-rt goodies were located in a path like ./lib/clang/7.0.0/lib/netbsd8.99.12 with a varying OS version. This could stop working after upgrades, so I've simplified it to "netbsd", similar to FreeBSD and Solaris.

  • Prebuilt toolchain for testers

I've prepared a build of Clang/LLVM with LLDB and compiler-rt features prebuilt on NetBSD/amd64 v. 8.99.12:

llvm-clang-compilerrt-lldb-7.0.0beta_2018-02-28.tar.bz2

  • Plan for the next milestone

With the approaching NetBSD 8.0 release I plan to finish backporting a few changes there from HEAD:

  • Remove one unused feature from ptrace(2), PTSETSIGMASK & PTGETSIGMASK. I've originally introduced these operations with criu/rr-like software in mind, but they are misusing or even abusing ptrace(2) and are not regular process debuggers. I plan to remove this operation from HEAD and backport this to NetBSD-8(BETA), before the release, so no compat will be required for this call. Future ports of criu/rr should involve dedicated kernel support for such requirements.
    Finish the backport of UCMACHINE_FP() to NetBSD-8. This will allow use of the same code in sanitizers in HEAD and NetBSD-8.0.
  • By popular demand, improve the regnsub(3) and regasub(3) API, adding support for more or less substitutions than 10.

Once done, I will return to ptrace(2) debugging and corrections.

DigitalOcean

Working with the NetBSD kernel

  • Overview

When working on complex systems, such as OS kernels, your attention span and cognitive energy are too valuable to be wasted on inefficiencies pertaining to ancillary tasks. After experimenting with different environmental setups for kernel debugging, some of which were awkward and distracting from my main objectives, I have arrived to my current workflow, which is described here. This approach is mainly oriented towards security research and the study of kernel internals.

Before delving into the details, this is the general outline of my environment:

My host system runs Linux. My target system is a QEMU guest.

I’m tracing and debugging on my host system by attaching GDB (with NetBSD x86-64 ABI support) to QEMU’s built-in GDB server.
I work with NetBSD-current. All sources are built on my host system with the cross-compilation toolchain produced by build.sh.
I use NFS to share the source tree and the build artifacts between the target and the host.
I find IDEs awkward, so for codebase navigation I mainly rely on vim, tmux and ctags.
For non-intrusive instrumentation, such as figuring out control flow, I’m using dtrace.

  • Preparing the host system

    • QEMU
    • GDB
    • NFS Exports
  • Building NetBSD-current

  • A word of warning

    • Now is a great time to familiarize yourself with the build.sh tool and its options. Be especially carefull with the following options:


-r Remove contents of TOOLDIR and DESTDIR before building.
-u Set MKUPDATE=yes; do not run "make clean" first.
Without this, everything is rebuilt, including the tools.

Chance are, you do not want to use these options once you’ve successfully built the cross-compilation toolchain and your entire userland, because building those takes time and there aren’t many good reasons to recompile them from scratch. Here’s what to expect:

On my desktop, running a quad-core Intel i5-3470 at 3.20GHz with 24GB of RAM and underlying directory structure residing on a SSD drive, the entire process took about 55 minutes. I was running make with -j12, so the machine was quite busy.
On an old Dell D630 laptop, running Intel Core 2 Duo T7500 at 2.20GHz with 4GB of RAM and a slow hard drive (5400RPM), the process took approximatelly 2.5 hours. I was running make with -j4. Based on the temperature alerts and CPU clock throttling messages, it was quite a struggle.

  • Acquiring the sources
  • Compiling the sources

    • Preparing the guest system
  • Provisioning your guest
  • Pkgin and NFS shares
  • Tailoring the kernel for debugging
  • Installing the new kernel
  • Configuring DTrace
  • Debugging the guest’s kernel

News Roundup

Add support for the experimental Internet-Draft "TCP Alternative Backoff”

```
Add support for the experimental Internet-Draft "TCP Alternative Backoff with
ECN (ABE)" proposal to the New Reno congestion control algorithm module.
ABE reduces the amount of congestion window reduction in response to
ECN-signalled congestion relative to the loss-inferred congestion response.

More details about ABE can be found in the Internet-Draft:
https://tools.ietf.org/html/draft-ietf-tcpm-alternativebackoff-ecn

The implementation introduces four new sysctls:

  • net.inet.tcp.cc.abe defaults to 0 (disabled) and can be set to non-zero to
    enable ABE for ECN-enabled TCP connections.

  • net.inet.tcp.cc.newreno.beta and net.inet.tcp.cc.newreno.betaecn set the
    multiplicative window decrease factor, specified as a percentage, applied to
    the congestion window in response to a loss-based or ECN-based congestion
    signal respectively. They default to the values specified in the draft i.e.
    beta=50 and beta
    ecn=80.

  • net.inet.tcp.cc.abe_frlossreduce defaults to 0 (disabled) and can be set to
    non-zero to enable the use of standard beta (50% by default) when repairing
    loss during an ECN-signalled congestion recovery episode. It enables a more
    conservative congestion response and is provided for the purposes of
    experimentation as a result of some discussion at IETF 100 in Singapore.

The values of beta and betaecn can also be set per-connection by way of the
TCP
CCALGOOPT TCP-level socket option and the new CCNEWRENOBETA or
CCNEWRENOBETA_ECN CC algo sub-options.

Submitted by: Tom Jones tj@enoti.me
Tested by: Tom Jones tj@enoti.me, Grenville Armitage garmitage@swin.edu.au
Relnotes: Yes
Differential Revision: https://reviews.freebsd.org/D11616
```

Meltdown-mitigation syspatch/errata now available

The recent changes in -current mitigating the Meltdown vulnerability have been backported to the 6.1 and 6.2 (amd64) releases, and the syspatch update (for 6.2) is now available.

```
Changes by: bluhm@cvs.openbsd.org 2018/02/26 05:36:18
Log message:
Implement a workaround against the Meltdown flaw in Intel CPUs.
The following changes have been backported from OpenBSD -current.

Changes by: guenther@cvs.openbsd.org 2018/01/06 15:03:13
Log message:
Handle %gs like %[def]s and reset set it in cpu_switchto() instead of on
every return to userspace.

Changes by: mlarkin@cvs.openbsd.org 2018/01/06 18:08:20
Log message:
Add identcpu.c and specialreg.h definitions for the new Intel/AMD MSRs
that should help mitigate spectre. This is just the detection piece, these
features are not yet used.
Part of a larger ongoing effort to mitigate meltdown/spectre. i386 will
come later; it needs some machdep.c cleanup first.

Changes by: mlarkin@cvs.openbsd.org 2018/01/07 12:56:19
Log message:
remove all PG_G global page mappings from the kernel when running on
Intel CPUs. Part of an ongoing set of commits to mitigate the Intel
"meltdown" CVE. This diff does not confer any immunity to that
vulnerability - subsequent commits are still needed and are being
worked on presently.
ok guenther, deraadt

Changes by: mlarkin@cvs.openbsd.org 2018/01/12 01:21:30
Log message:
IBRS -> IBRS,IBPB in identifycpu lines

Changes by: guenther@cvs.openbsd.org 2018/02/21 12:24:15
Log message:
Meltdown: implement user/kernel page table separation.
On Intel CPUs which speculate past user/supervisor page permission checks,
use a separate page table for userspace with only the minimum of kernel code
and data required for the transitions to/from the kernel (still marked as
supervisor-only, of course):
- the IDT (RO)
- three pages of kernel text in the .kutext section for interrupt, trap,
and syscall trampoline code (RX)
- one page of kernel data in the .kudata section for TLB flush IPIs (RW)
- the lapic page (RW, uncachable)
- per CPU: one page for the TSS+GDT (RO) and one page for trampoline
stacks (RW)
When a syscall, trap, or interrupt takes a CPU from userspace to kernel the
trampoline code switches page tables, switches stacks to the thread's real
kernel stack, then copies over the necessary bits from the trampoline stack.
On return to userspace the opposite occurs: recreate the iretq frame on the
trampoline stack, switch stack, switch page tables, and return to userspace.
mlarkin@ implemented the pmap bits and did 90% of the debugging, diagnosing
issues on MP in particular, and drove the final push to completion.
Many rounds of testing by naddy@, sthen@, and others
Thanks to Alex Wilson from Joyent for early discussions about trampolines
and their data requirements.
Per-CPU page layout mostly inspired by DragonFlyBSD.
ok mlarkin@ deraadt@

Changes by: bluhm@cvs.openbsd.org 2018/02/22 13:18:59
Log message:
The GNU assembler does not understand 1ULL, so replace the constant
with 1. Then it compiles with gcc, sign and size do not matter
here.

Changes by: bluhm@cvs.openbsd.org 2018/02/22 13:27:14
Log message:
The compile time assertion for cpu info did not work with gcc.
Rephrase the condition in a way that both gcc and clang accept it.

Changes by: guenther@cvs.openbsd.org 2018/02/22 13:36:40
Log message:
Set the PG_G (global) bit on the special page table entries that are shared
between the u-k and u+k tables, because they're actually in all tables.

OpenBSD 6.1 errata 037
```

  • 6.2

```
Changes by: bluhm@cvs.openbsd.org 2018/02/26 05:29:48
Log message:
Implement a workaround against the Meltdown flaw in Intel CPUs.
The following changes have been backported from OpenBSD -current.

Changes by: guenther@cvs.openbsd.org 2018/01/06 15:03:13
Log message:
Handle %gs like %[def]s and reset set it in cpu_switchto() instead of on
every return to userspace.

Changes by: mlarkin@cvs.openbsd.org 2018/01/06 18:08:20
Log message:
Add identcpu.c and specialreg.h definitions for the new Intel/AMD MSRs
that should help mitigate spectre. This is just the detection piece, these
features are not yet used.
Part of a larger ongoing effort to mitigate meltdown/spectre. i386 will
come later; it needs some machdep.c cleanup first.

Changes by: mlarkin@cvs.openbsd.org 2018/01/07 12:56:19
Log message:
remove all PG_G global page mappings from the kernel when running on
Intel CPUs. Part of an ongoing set of commits to mitigate the Intel
"meltdown" CVE. This diff does not confer any immunity to that
vulnerability - subsequent commits are still needed and are being
worked on presently.

Changes by: mlarkin@cvs.openbsd.org 2018/01/12 01:21:30
Log message:
IBRS -> IBRS,IBPB in identifycpu lines

Changes by: guenther@cvs.openbsd.org 2018/02/21 12:24:15
Log message:
Meltdown: implement user/kernel page table separation.
On Intel CPUs which speculate past user/supervisor page permission checks,
use a separate page table for userspace with only the minimum of kernel code
and data required for the transitions to/from the kernel (still marked as
supervisor-only, of course):
- the IDT (RO)
- three pages of kernel text in the .kutext section for interrupt, trap,
and syscall trampoline code (RX)
- one page of kernel data in the .kudata section for TLB flush IPIs (RW)
- the lapic page (RW, uncachable)
- per CPU: one page for the TSS+GDT (RO) and one page for trampoline
stacks (RW)
When a syscall, trap, or interrupt takes a CPU from userspace to kernel the
trampoline code switches page tables, switches stacks to the thread's real
kernel stack, then copies over the necessary bits from the trampoline stack.
On return to userspace the opposite occurs: recreate the iretq frame on the
trampoline stack, switch stack, switch page tables, and return to userspace.
mlarkin@ implemented the pmap bits and did 90% of the debugging, diagnosing
issues on MP in particular, and drove the final push to completion.
Many rounds of testing by naddy@, sthen@, and others
Thanks to Alex Wilson from Joyent for early discussions about trampolines
and their data requirements.
Per-CPU page layout mostly inspired by DragonFlyBSD.

Changes by: bluhm@cvs.openbsd.org 2018/02/22 13:18:59
Log message:
The GNU assembler does not understand 1ULL, so replace the constant
with 1. Then it compiles with gcc, sign and size do not matter
here.

Changes by: bluhm@cvs.openbsd.org 2018/02/22 13:27:14
Log message:
The compile time assertion for cpu info did not work with gcc.
Rephrase the condition in a way that both gcc and clang accept it.

Changes by: guenther@cvs.openbsd.org 2018/02/22 13:36:40
Log message:
Set the PG_G (global) bit on the special page table entries that are shared
between the u-k and u+k tables, because they're actually in all tables.

OpenBSD 6.2 errata 009
```

iXsystems

a2k18 Hackathon Report: Ken Westerback on dhclient and more

Ken Westerback (krw@) has sent in the first report from the (recently concluded) a2k18 hackathon:

```

Whew.

Once in Dunedin the hacking commenced. The background was a regular tick of new meltdown diffs to test in addition to whatever work one was actually engaged in. I was lucky (?) in that none of the problems with the various versions cropped up on my laptop.
```

```
I worked with rpe@ and tb@ to make the install script create the 'correct' FQDN when dhclient was involved. I worked with tb@ on some code cleanup in various bits of the base. dhclient(8) got some nice cleanup, further pruning/improving log messages in particular. In addition the oddball -q option was flipped into the more normal -v. I.e. be quiet by default and verbose on request.

More substantially the use of recorded leases was made less intrusive by avoiding continual reconfiguration of the interface with the same information. The 'request', 'require' and 'ignore' dhclient.conf(5) statement were changed so they are cumulative, making it easier to build longer lists of affected options.

I tweaked softraid(4) to remove a handrolled version of duid_format().

I sprinkled a couple of M_WAITOK into amd64 and i386 mpbios to document that there is really no need to check for NULL being returned from some malloc() calls.

I continued to help test the new filesystem quiescing logic that deraadt@ committed during the hackathon.

I only locked myself out of my room once!

Fueled by the excellent coffee from local institutions The Good Earth Cafe and The Good Oil Cafe, and the excellent hacking facilities and accommodations at the University of Otago it was another enjoyable and productive hackathon south of the equator. And I even saw penguins.

Thanks to Jim Cheetham and the support from the project and the OpenBSD Foundation that made it all possible
```

Poetic License

I found this when going through old documents. It looks like I wrote it and never posted it. Perhaps I didn’t consider it finished at the time. But looking at it now, I think it’s good enough to share. It’s a redrafting of the BSD licence, in poetic form. Maybe I had plans to do other licences one day; I can’t remember.

I’ve interleaved it with the original license text so you can see how true, or otherwise, I’ve been to it. Enjoy :-)

```
Copyright (c) ,
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
```

You may redistribute and use –
as source or binary, as you choose,
and with some changes or without –
this software; let there be no doubt.
But you must meet conditions three,
if in compliance you wish to be.


1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. Neither the name of the nor the names of its
contributors may be used to endorse or promote products derived
from this software without specific prior written permission.

The first is obvious, of course –
To keep this text within the source.
The second is for binaries
Place in the docs a copy, please.
A moral lesson from this ode –
Don’t strip the copyright on code.

The third applies when you promote:
You must not take, from us who wrote,
our names and make it seem as true
we like or love your version too.
(Unless, of course, you contact us
And get our written assensus.)


THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.

One final point to be laid out
(You must forgive my need to shout):
THERE IS NO WARRANTY FOR THIS
WHATEVER THING MAY GO AMISS.
EXPRESS, IMPLIED, IT’S ALL THE SAME –
RESPONSIBILITY DISCLAIMED.

WE ARE NOT LIABLE FOR LOSS
NO MATTER HOW INCURRED THE COST
THE TYPE OR STYLE OF DAMAGE DONE
WHATE’ER THE LEGAL THEORY SPUN.
THIS STILL REMAINS AS TRUE IF YOU
INFORM US WHAT YOU PLAN TO DO.

When all is told, we sum up thus –
Do what you like, just don’t sue us.

Beastie Bits

Tarsnap ad

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Mar 29 2018

1hr 32mins

Play

Rank #4: 345: Switchers to BSD

Podcast cover
Read more

NetBSD 8.2 is available, NextCloud on OpenBSD, X11 screen locking, NetBSD and RISC OS running parallel, community feedback about switching to BSD, and more.

Headlines

NetBSD 8.2 is available!

The third release in the NetBSD-8 is now available.

This release includes all the security fixes in NetBSD-8 up until this point, and other fixes deemed important for stability.

  • Some highlights include:
    • x86: fixed regression in booting old CPUs
    • x86: Hyper-V Gen.2 VM framebuffer support
    • httpd(8): fixed various security issues
    • ixg(4): various fixes / improvements
    • x86 efiboot: add tftp support, fix issues on machines with many memory segments, improve graphics mode logic to work on more machines.
    • Various kernel memory info leaks fixes
    • Update expat to 2.2.8
    • Fix ryzen USB issues and support xHCI version 3.10.
    • Accept root device specification as NAME=label.
    • Add multiboot 2 support to x86 bootloaders.
    • Fix for CVE-2019-9506: 'Key Negotiation of Bluetooth' attack.
    • nouveau: limit the supported devices and fix firmware loading.
    • radeon: fix loading of the TAHITI VCE firmware.
    • named(8): stop using obsolete dnssec-lookaside.

NextCloud on OpenBSD

NextCloud and OpenBSD are complementary to one another. NextCloud is an awesome, secure and private alternative for proprietary platforms, whereas OpenBSD forms the most secure and solid foundation to serve it on. Setting it up in the best way isn’t hard, especially using this step by step tutorial.

  • Preface

Back when this tutorial was initially written, things were different. The OpenBSD port relied on PHP 5.6 and there were no package updates. But the port improved (hats off, Gonzalo!) and package updates were introduced to the -stable branch (hats off, Solene!).

A rewrite of this tutorial was long overdue. Right now, it is written for 6.6 -stable and will be updated once 6.7 is released. If you have any questions or desire some help, feel free to reach out.

News Roundup

X11 screen locking: a secure and modular approach

For years I’ve been using XScreenSaver as a default, but I recently learned about xsecurelock and re-evaluated my screen-saving requirements

NetBSD and RISC OS running parallel

I have been experimenting with running two systems at the same time on the RK3399 SoC.
It all begun when I figured out how to switch to the A72 cpu for RISC OS. When the switch was done, the A53 cpu just continued to execute code.
OK I thought why not give it something to do!
My first step was to run some small programs.
It worked!

  • Thanks to Tom Jones for the pointer to this article

Several weeks ago we covered a story about switching from Linux to BSD. Benedict and JT asked for community feedback as to their thoughts on the matter. Allan was out that week, so this will give him an opportunity to chime in with his thoughts as well.

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Your browser does not support the HTML5 video tag.

Apr 09 2020

47mins

Play

Rank #5: 344: Grains of Salt

Podcast cover
Read more

Shell text processing, data rebalancing on ZFS mirrors, Add Security Headers with OpenBSD relayd, ZFS filesystem hierarchy in ZFS pools, speeding up ZSH, How Unix pipes work, grow ZFS pools over time, the real reason ifconfig on Linux is deprecated, clear your terminal in style, and more.

Headlines

Text processing in the shell

This article is part of a self-published book project by Balthazar Rouberol and Etienne Brodu, ex-roommates, friends and colleagues, aiming at empowering the up and coming generation of developers. We currently are hard at work on it!

One of the things that makes the shell an invaluable tool is the amount of available text processing commands, and the ability to easily pipe them into each other to build complex text processing workflows. These commands can make it trivial to perform text and data analysis, convert data between different formats, filter lines, etc.

When working with text data, the philosophy is to break any complex problem you have into a set of smaller ones, and to solve each of them with a specialized tool.

Rebalancing data on ZFS mirrors

One of the questions that comes up time and time again about ZFS is “how can I migrate my data to a pool on a few of my disks, then add the rest of the disks afterward?”

If you just want to get the data moved and don’t care about balance, you can just copy the data over, then add the new disks and be done with it. But, it won’t be distributed evenly over the vdevs in your pool.

Don’t fret, though, it’s actually pretty easy to rebalance mirrors. In the following example, we’ll assume you’ve got four disks in a RAID array on an old machine, and two disks available to copy the data to in the short term.

News Roundup

Using OpenBSD relayd to Add Security Headers

I am a huge fan of OpenBSD’s built-in httpd server as it is simple, secure, and quite performant. With the modern push of the large search providers pushing secure websites, it is now important to add security headers to your website or risk having the search results for your website downgraded. Fortunately, it is very easy to do this when you combine httpd with relayd. While relayd is principally designed for layer 3 redirections and layer 7 relays, it just so happens that it makes a handy tool for adding the recommended security headers. My website automatically redirects users from http to https and this gets achieved using a simple redirection in /etc/httpd.conf So if you have a configuration similar to mine, then you will still want to have httpd listen on the egress interface on port 80. The key thing to change here is to have httpd listen on 127.0.0.1 on port 443.

How we set up our ZFS filesystem hierarchy in our ZFS pools

Our long standing practice here, predating even the first generation of our ZFS fileservers, is that we have two main sorts of filesystems, home directories (homedir filesystems) and what we call 'work directory' (workdir) filesystems. Homedir filesystems are called /h/NNN (for some NNN) and workdir filesystems are called /w/NNN; the NNN is unique across all of the different sorts of filesystems. Users are encouraged to put as much stuff as possible in workdirs and can have as many of them as they want, which mattered a lot more in the days when we used Solaris DiskSuite and had fixed-sized filesystems.

Speeding up ZSH

https://web.archive.org/web/20200315184849/https://blog.jonlu.ca/posts/speeding-up-zsh

I was opening multiple shells for an unrelated project today and noticed how abysmal my shell load speed was. After the initial load it was relatively fast, but the actual shell start up was noticeably slow. I timed it with time and these were the results.

In the future I hope to actually recompile zsh with additional profiling techniques and debug information - keeping an internal timer and having a flag output current time for each command in a tree fashion would make building heat maps really easy.

How do Unix Pipes work

Pipes are cool! We saw how handy they are in a previous blog post. Let’s look at a typical way to use the pipe operator. We have some output, and we want to look at the first lines of the output. Let’s download The Brothers Karamazov by Fyodor Dostoevsky, a fairly long novel.

What we do to enable us to grow our ZFS pools over time

In my entry on why ZFS isn't good at growing and reshaping pools, I mentioned that we go to quite some lengths in our ZFS environment to be able to incrementally expand our pools. Today I want to put together all of the pieces of that in one place to discuss what those lengths are.
Our big constraint is that not only do we need to add space to pools over time, but we have a fairly large number of pools and which pools will have space added to them is unpredictable. We need a solution to pool expansion that leaves us with as much flexibility as possible for as long as possible. This pretty much requires being able to expand pools in relatively small increments of space.

Linux maintains bugs: The real reason ifconfig on Linux is deprecated

In my third installment of FreeBSD vs Linux, I will discuss underlying reasons for why Linux moved away from ifconfig(8) to ip(8).

In the past, when people said, “Linux is a kernel, not an operating system”, I knew that was true but I always thought it was a rather pedantic criticism. Of course no one runs just the Linux kernel, you run a distribution of Linux. But after reviewing userland code, I understand the significant drawbacks to developing “just a kernel” in isolation from the rest of the system.

Clear Your Terminal in Style

if you’re someone like me who habitually clears their terminal, sometimes you want a little excitement in your life. Here is a way to do just that.

This post revolves around the idea of giving a command a percent chance of running. While the topic at hand is not serious, this simple technique has potential in your scripts.

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Your browser does not support the HTML5 video tag.

Apr 02 2020

55mins

Play

Rank #6: Episode 244: C is a Lie | BSD Now 244

Podcast cover
Read more

Arcan and OpenBSD, running OpenBSD 6.3 on RPI 3, why C is not a low-level language, HardenedBSD switching back to OpenSSL, how the Internet was almost broken, EuroBSDcon CfP is out, and the BSDCan 2018 schedule is available.

Headlines

Towards Secure System Graphics: Arcan and OpenBSD

Let me preface this by saying that this is a (very) long and medium-rare technical article about the security considerations and minutiae of porting (most of) the Arcan ecosystem to work under OpenBSD. The main point of this article is not so much flirting with the OpenBSD crowd or adding further noise to software engineering topics, but to go through the special considerations that had to be taken, as notes to anyone else that decides to go down this overgrown and lonesome trail, or are curious about some less than obvious differences between how these things “work” on Linux vs. other parts of the world.

A disclaimer is also that most of this have been discovered by experimentation and combining bits and pieces scattered in everything from Xorg code to man pages, there may be smarter ways to solve some of the problems mentioned – this is just the best I could find within the time allotted. I’d be happy to be corrected, in patch/pull request form that is 😉

Each section will start with a short rant-like explanation of how it works in Linux, and what the translation to OpenBSD involved or, in the cases that are still partly or fully missing, will require. The topics that will be covered this time are:

  • Graphics Device Access
  • Hotplug
  • Input
  • Backlight
  • Xorg
  • Pledging
  • Missing

Installing OpenBSD 6.3 (snapshots) on Raspberry pi 3

  • The Easy way

Installing the OpenBSD on raspberry pi 3 is very easy and well documented which almost convinced me of not writing about it, but still I felt like it may help somebody new to the project (But again I really recommend reading the document if you are interested and have the time).

Note: I'm always running snapshots and recommend anybody to do it as well. But the snapshots links will change to the next version every 6 month, so I changed the links to the 6.3 version to keep the blog post valid over times. If you're familiar to the OpenBSD flavors, feel free to use the snapshots links instead.

  • Requirements

Due to the lack of driver, the OpenBSD can not boot directly from the SD Card yet, So we'll need an USB Stick for the installtion target aside the SD Card for the U-Boot and installer. Also, a Serial Console connection is required. I Used a PL2303 USB to Serial (TTL) adapter connected to my Laptop via USB port and connected to the Raspberry via TX, RX and GND pins.

iXsystems https://www.ixsystems.com/blog/truenas-m-series-veeam-pr-2018/

Why Didn’t Larrabee Fail?

Every month or so, someone will ask me what happened to Larrabee and why it failed so badly. And I then try to explain to them that not only didn't it fail, it was a pretty huge success. And they are understandably very puzzled by this, because in the public consciousness Larrabee was like the Itanic and the SPU rolled into one, wasn't it? Well, not quite. So rather than explain it in person a whole bunch more times, I thought I should write it down.

This is not a history, and I'm going to skip a TON of details for brevity. One day I'll write the whole story down, because it's a pretty decent escapade with lots of fun characters. But not today. Today you just get the very start and the very end.

When I say "Larrabee" I mean all of Knights, all of MIC, all of Xeon Phi, all of the "Isle" cards - they're all exactly the same chip and the same people and the same software effort. Marketing seemed to dream up a new codeword every week, but there was only ever three chips:

  • Knights Ferry / Aubrey Isle / LRB1 - mostly a prototype, had some performance gotchas, but did work, and shipped to partners.
  • Knights Corner / Xeon Phi / LRB2 - the thing we actually shipped in bulk.
  • Knights Landing - the new version that is shipping any day now (mid 2016).

That's it. There were some other codenames I've forgotten over the years, but they're all of one of the above chips. Behind all the marketing smoke and mirrors there were only three chips ever made (so far), and only four planned in total (we had a thing called LRB3 planned between KNC and KNL for a while). All of them are "Larrabee", whether they do graphics or not.

When Larrabee was originally conceived back in about 2005, it was called "SMAC", and its original goals were, from most to least important:

    1. Make the most powerful flops-per-watt machine for real-world workloads using a huge array of simple cores, on systems and boards that could be built into bazillo-core supercomputers.
    1. Make it from x86 cores. That means memory coherency, store ordering, memory protection, real OSes, no ugly scratchpads, it runs legacy code, and so on. No funky DSPs or windowed register files or wacky programming models allowed. Do not build another Itanium or SPU!
    1. Make it soon. That means keeping it simple.
    1. Support the emerging GPGPU market with that same chip. Intel were absolutely not going to build a 150W PCIe card version of their embedded graphics chip (known as "Gen"), so we had to cover those programming models. As a bonus, run normal graphics well.
    1. Add as little graphics-specific hardware as you can get away with.

That ordering is important - in terms of engineering and focus, Larrabee was never primarily a graphics card. If Intel had wanted a kick-ass graphics card, they already had a very good graphics team begging to be allowed to build a nice big fat hot discrete GPU - and the Gen architecture is such that they'd build a great one, too. But Intel management didn't want one, and still doesn't. But if we were going to build Larrabee anyway, they wanted us to cover that market as well.

... the design of Larrabee was of a CPU with a very wide SIMD unit, designed above all to be a real grown-up CPU - coherent caches, well-ordered memory rules, good memory protection, true multitasking, real threads, runs Linux/FreeBSD, etc. Larrabee, in the form of KNC, went on to become the fastest supercomputer in the world for a couple of years, and it's still making a ton of money for Intel in the HPC market that it was designed for, fighting very nicely against the GPUs and other custom architectures. Its successor, KNL, is just being released right now (mid 2016) and should do very nicely in that space too. Remember - KNC is literally the same chip as LRB2. It has texture samplers and a video out port sitting on the die. They don't test them or turn them on or expose them to software, but they're still there - it's still a graphics-capable part.

But it's still actually running FreeBSD on that card, and under FreeBSD it's just running an x86 program called DirectXGfx (248 threads of it).

News Roundup

C Is Not a Low-level Language : Your computer is not a fast PDP-11.

In the wake of the recent Meltdown and Spectre vulnerabilities, it's worth spending some time looking at root causes. Both of these vulnerabilities involved processors speculatively executing instructions past some kind of access check and allowing the attacker to observe the results via a side channel. The features that led to these vulnerabilities, along with several others, were added to let C programmers continue to believe they were programming in a low-level language, when this hasn't been the case for decades.

Processor vendors are not alone in this. Those of us working on C/C++ compilers have also participated.

  • What Is a Low-Level Language?

Computer science pioneer Alan Perlis defined low-level languages this way: "A programming language is low level when its programs require attention to the irrelevant."

While, yes, this definition applies to C, it does not capture what people desire in a low-level language. Various attributes cause people to regard a language as low-level. Think of programming languages as belonging on a continuum, with assembly at one end and the interface to the Starship Enterprise's computer at the other. Low-level languages are "close to the metal," whereas high-level languages are closer to how humans think.

For a language to be "close to the metal," it must provide an abstract machine that maps easily to the abstractions exposed by the target platform. It's easy to argue that C was a low-level language for the PDP-11. They both described a model in which programs executed sequentially, in which memory was a flat space, and even the pre- and post-increment operators cleanly lined up with the PDP-11 addressing modes.

Fast PDP-11 Emulators

The root cause of the Spectre and Meltdown vulnerabilities was that processor architects were trying to build not just fast processors, but fast processors that expose the same abstract machine as a PDP-11. This is essential because it allows C programmers to continue in the belief that their language is close to the underlying hardware.

C code provides a mostly serial abstract machine (until C11, an entirely serial machine if nonstandard vendor extensions were excluded). Creating a new thread is a library operation known to be expensive, so processors wishing to keep their execution units busy running C code rely on ILP (instruction-level parallelism). They inspect adjacent operations and issue independent ones in parallel. This adds a significant amount of complexity (and power consumption) to allow programmers to write mostly sequential code. In contrast, GPUs achieve very high performance without any of this logic, at the expense of requiring explicitly parallel programs.

The quest for high ILP was the direct cause of Spectre and Meltdown. A modern Intel processor has up to 180 instructions in flight at a time (in stark contrast to a sequential C abstract machine, which expects each operation to complete before the next one begins). A typical heuristic for C code is that there is a branch, on average, every seven instructions. If you wish to keep such a pipeline full from a single thread, then you must guess the targets of the next 25 branches. This, again, adds complexity; it also means that an incorrect guess results in work being done and then discarded, which is not ideal for power consumption. This discarded work has visible side effects, which the Spectre and Meltdown attacks could exploit.

On a modern high-end core, the register rename engine is one of the largest consumers of die area and power. To make matters worse, it cannot be turned off or power gated while any instructions are running, which makes it inconvenient in a dark silicon era when transistors are cheap but powered transistors are an expensive resource. This unit is conspicuously absent on GPUs, where parallelism again comes from multiple threads rather than trying to extract instruction-level parallelism from intrinsically scalar code. If instructions do not have dependencies that need to be reordered, then register renaming is not necessary.

Consider another core part of the C abstract machine's memory model: flat memory. This hasn't been true for more than two decades. A modern processor often has three levels of cache in between registers and main memory, which attempt to hide latency.

The cache is, as its name implies, hidden from the programmer and so is not visible to C. Efficient use of the cache is one of the most important ways of making code run quickly on a modern processor, yet this is completely hidden by the abstract machine, and programmers must rely on knowing implementation details of the cache (for example, two values that are 64-byte-aligned may end up in the same cache line) to write efficient code.

HardenedBSD Switching Back to OpenSSL

Over a year ago, HardenedBSD switched to LibreSSL as the default cryptographic library in base for 12-CURRENT. 11-STABLE followed suit later on. Bernard Spil has done an excellent job at keeping our users up-to-date with the latest security patches from LibreSSL.

After recently updating 12-CURRENT to LibreSSL 2.7.2 from 2.6.4, it has become increasingly clear to us that performing major upgrades requires a team larger than a single person. Upgrading to 2.7.2 caused a lot of fallout in our ports tree. As of 28 Apr 2018, several ports we consider high priority are still broken. As it stands right now, it would take Bernard a significant amount of his spare personal time to fix these issues.

Until we have a multi-person team dedicated to maintaining LibreSSL in base along with the patches required in ports, HardenedBSD will use OpenSSL going forward as the default cryptographic library in base. LibreSSL will co-exist with OpenSSL in the source tree, as it does now. However, MK_LIBRESSL will default to "no" instead of the current "yes". Bernard will continue maintaining LibreSSL in base along with addressing the various problematic ports entries.

To provide our users with ample time to plan and perform updates, we will wait a period of two months prior to making the switch. The switch will occur on 01 Jul 2018 and will be performed simultaneously in 12-CURRENT and 11-STABLE. HardenedBSD will archive a copy of the LibreSSL-centric package repositories and binary updates for base for a period of six months after the switch (expiring the package repos on 01 Jan 2019). This essentially gives our users eight full months for an upgrade path.

As part of the switch back to OpenSSL, the default NTP daemon in base will switch back from OpenNTPd to ISC NTP. Users who have localopenntpdenable="YES" set in rc.conf will need to switch back to ntpd_enable="YES".

Users who build base from source will want to fully clean their object directories. Any and all packages that link with libcrypto or libssl will need to be rebuilt or reinstalled.

With the community's help, we look forward to the day when we can make the switch back to LibreSSL. We at HardenedBSD believe that providing our users options to rid themselves of software monocultures can better increase security and manage risk.

DigitalOcean http://do.co/bsdnow -- $100 credit for 60 days

How Dan Kaminsky Almost Broke the Internet

In the summer of 2008, security researcher Dan Kaminsky disclosed how he had found a huge flaw in the Internet that could let attackers redirect web traffic to alternate servers and disrupt normal operations. In this Hacker History video, Kaminsky describes the flaw and notes the issue remains unfixed.

“We were really concerned about web pages and emails 'cause that’s what you get to compromise when you compromise DNS,” Kaminsky says. “You think you’re sending an email to IBM but it really goes to the bad guy.”

As the phone book of the Internet, DNS translates easy-to-remember domain names into IP addresses so that users don’t have to remember strings of numbers to reach web applications and services. Authoritative nameservers publish the IP addresses of domain names. Recursive nameservers talk to authoritative servers to find addresses for those domain names and saves the information into its cache to speed up the response time the next time it is asked about that site. While anyone can set up a nameserver and configure an authoritative zone for any site, if recursive nameservers don’t point to it to ask questions, no one will get those wrong answers.

We made the Internet less flammable.

Kaminsky found a fundamental design flaw in DNS that made it possible to inject incorrect information into the nameserver's cache, or DNS cache poisoning. In this case, if an attacker crafted DNS queries looking for sibling names to existing domains, such as 1.example.com, 2.example.com, and 3.example.com, while claiming to be the official "www" server for example.com, the nameserver will save that server IP address for “www” in its cache.

“The server will go, ‘You are the official. Go right ahead. Tell me what it’s supposed to be,’” Kaminsky says in the video.

Since the issue affected nearly every DNS server on the planet, it required a coordinated response to address it. Kaminsky informed Paul Vixie, creator of several DNS protocol extensions and application, and Vixie called an emergency summit of major IT vendors at Microsoft’s headquarters to figure out what to do.

The “fix” involved combining the 16-bit transaction identifier that DNS lookups used with UDP source ports to create 32-bit transaction identifiers. Instead of fixing the flaw so that it can’t be exploited, the resolution focused on making it take more than ten seconds, eliminating the instantaneous attack.

“[It’s] not like we repaired DNS,” Kaminsky says. “We made the Internet less flammable.”

DNSSEC (Domain Name System Security Extensions), is intended to secure DNS by adding a cryptographic layer to DNS information. The root zone of the internet was signed for DNSSEC in July 2010 and the .com Top Level Domain (TLD) was finally signed for DNSSEC in April 2011. Unfortunately, adoption has been slow, even ten years after Kaminsky first raised the alarm about DNS, as less than 15 percent of users pass their queries to DNSSEC validating resolvers.

The Internet was never designed to be secure. The Internet was designed to move pictures of cats.

No one expected the Internet to be used for commerce and critical communications. If people lose faith in DNS, then all the things that depend on it are at risk.

“What are we going to do? Here is the answer. Some of us gotta go out fix it,” Kaminsky says.

OpenIndiana Hipster 2018.04 is here

  • We have released a new OpenIndiana Hipster snapshot 2018.04. The noticeable changes:

    • Userland software is rebuilt with GCC 6.
    • KPTI was enabled to mitigate recent security issues in Intel CPUs.
    • Support of Gnome 2 desktop was removed.
    • Linked images now support zoneproxy service.
    • Mate desktop applications are delivered as 64-bit-only.
    • Upower support was integrated.
    • IIIM was removed.
  • More information can be found in 2018.04 Release notes and new medias can be downloaded from http://dlc.openindiana.org.

Beastie Bits

Tarsnap ad

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

iX Ad spot: iXsystems TrueNAS M-Series Blows Away Veeam Backup Certification Tests

May 03 2018

1hr 25mins

Play

Rank #7: 296: It’s Alive: OpenBSD 6.5

Podcast cover
Read more

OpenBSD 6.5 has been released, mount ZFS datasets anywhere, help test upcoming NetBSD 9 branch, LibreSSL 2.9.1 is available, Bail Bond Denied Edition of FreeBSD Mastery: Jails, and one reason ed(1) was a good editor back in the days in this week’s episode.

Headlines

OpenBSD 6.5 Released

  • Changelog
  • Mirrors
  • 6.5 Includes
    • OpenSMTPD 6.5.0
    • LibreSSL 2.9.1
    • OpenSSH 8.0
    • Mandoc 1.14.5
    • Xenocara
    • LLVM/Clang 7.0.1 (+ patches)
    • GCC 4.2.1 (+ patches) and 3.3.6 (+ patches)
  • Many pre-built packages for each architecture:
    • aarch64: 9654
    • amd64: 10602
    • i386: 10535

Mount your ZFS datasets anywhere you want

ZFS is very flexible about mountpoints, and there are many features available to provide great flexibility.
When you create zpool maintank, the default mountpoint is /maintank.
You might be happy with that, but you don’t have to be content. You can do magical things.

  • Some highlights are:
    • mount point can be inherited
    • not all filesystems in a zpool need to be mounted
    • each filesystem (directory) can have different ZFS characteristics
    • In my case, let’s look at this new zpool I created earlier today and I will show you some very simple alternatives. This zpool use NVMe devices which should be faster than SSDs especially when used with multiple concurrent writes. This is my plan: run all the Bacula regression tests concurrently.

News Roundup

Branch for netbsd 9 upcoming, please help and test -current

Folks,
once again we are quite late for branching the next NetBSD release (NetBSD 9).
Initially planned to happen early in February 2019, we are now approaching May and it is unlikely that the branch will happen before that.
On the positive side, lots of good things landed in -current in between, like new Mesa, new jemalloc, lots of ZFS improvements - and some of those would be hard to pull up to the branch later.
On the bad side we saw lots of churn in -current recently, and there is quite some fallout where we not even have a good overview right now. And this is where you can help:

  • please test -current, on all the various machines you have
  • especially interesting would be test results from uncommon architectures
    or strange combinations (like the sparc userland on sparc64 kernel issue
    I ran in yesterday)
    Please test, report success, and file PRs for failures!
    We will likely announce the real branch date on quite short notice, the likely next candidates would be mid may or end of may.
    We may need to do extra steps after the branch (like switch some architectures back to old jemalloc on the branch). However, the less difference between -current and the branch, the easier will the release cycle go.
    Our goal is to have an unprecedented short release cycle this time. But..
    we always say that upfront.

LibreSSL 2.9.1 Released

We have released LibreSSL 2.9.1, which will be arriving in the LibreSSL
directory of your local OpenBSD mirror soon. This is the first stable release
from the 2.9 series, which is also included with OpenBSD 6.5

It includes the following changes and improvements from LibreSSL 2.8.x:

  • API and Documentation Enhancements

    • CRYPTO_LOCK is now automatically initialized, with the legacy
      callbacks stubbed for compatibility.
    • Added the SM3 hash function from the Chinese standard GB/T 32905-2016.
    • Added the SM4 block cipher from the Chinese standard GB/T 32907-2016.
    • Added more OPENSSLNO* macros for compatibility with OpenSSL.
    • Partial port of the OpenSSL ECKEYMETHOD API for use by OpenSSH.
    • Implemented further missing OpenSSL 1.1 API.
    • Added support for XChaCha20 and XChaCha20-Poly1305.
    • Added support for AES key wrap constructions via the EVP interface.
  • Compatibility Changes

    • Added pbkdf2 key derivation support to openssl(1) enc.
    • Changed the default digest type of openssl(1) enc to sha256.
    • Changed the default digest type of openssl(1) dgst to sha256.
    • Changed the default digest type of openssl(1) x509 -fingerprint to sha256.
    • Changed the default digest type of openssl(1) crl -fingerprint to sha256.
  • Testing and Proactive Security

    • Added extensive interoperability tests between LibreSSL and OpenSSL
      1.0 and 1.1.
    • Added additional Wycheproof tests and related bug fixes.
  • Internal Improvements

    • Simplified sigalgs option processing and handshake signing
      algorithm selection.
    • Added the ability to use the RSA PSS algorithm for handshake signatures.
    • Added bnrandinterval() and use it in code needing ranges of
      random bn values.
    • Added functionality to derive early, handshake, and application
      secrets as per RFC8446.
    • Added handshake state machine from RFC8446.
    • Removed some ASN.1 related code from libcrypto that had not been
      used since around 2000.
    • Unexported internal symbols and internalized more record layer structs.
    • Removed SHA224 based handshake signatures from consideration for
      use in a TLS 1.2 handshake.
  • Portable Improvements

    • Added support for assembly optimizations on 32-bit ARM ELF targets.
    • Added support for assembly optimizations on Mingw-w64 targets.
    • Improved Android compatibility
  • Bug Fixes

    • Improved protection against timing side channels in ECDSA signature
      generation.
    • Coordinate blinding was added to some elliptic curves. This is the
      last bit of the work by Brumley et al. to protect against the Portsmash
      vulnerability.
    • Ensure transcript handshake is always freed with TLS 1.2.

The LibreSSL project continues improvement of the codebase to reflect modern,
safe programming practices. We welcome feedback and improvements from the
broader community. Thanks to all of the contributors who helped make this
release possible.

FreeBSD Mastery: Jails – Bail Bond Denied Edition

I had a brilliant, hideous idea: to produce a charity edition of FreeBSD Mastery: Jails featuring the cover art I would use if I was imprisoned and did not have access to a real cover artist. (Never mind that I wouldn’t be permitted to release books while in jail: we creative sorts scoff at mere legal and cultural details.)
I originally wanted to produce my own take on the book’s cover art. My first attempt failed spectacularly.
I downgraded my expectations and tried again. And again. And again.
I’m pleased to reveal the final cover for FreeBSD Mastery: Jails–Bail Bond Edition!
This cover represents the very pinnacle of my artistic talents, and is the result of literally hours of effort.
But, as this book is available only to the winner of charity fund-raisers, purchase of this tome represents moral supremacy. I recommend flaunting it to your family, coworkers, and all those of lesser character.
Get your copy by winning the BSDCan 2019 charity auction… or any other other auction-type event I deem worthwhile.
As far as my moral fiber goes: I have learned that art is hard, and that artists are not paid enough.
And if I am ever imprisoned, I do hope that you’ll contribute to my bail fund. Otherwise, you’ll get more covers like this one.

One reason ed(1) was a good editor back in the days of V7 Unix

It is common to describe ed(1) as being line oriented, as opposed to screen oriented editors like vi. This is completely accurate but it is perhaps not a complete enough description for today, because ed is line oriented in a way that is now uncommon. After all, you could say that your shell is line oriented too, and very few people use shells that work and feel the same way ed does.
The surface difference between most people's shells and ed is that most people's shells have some version of cursor based interactive editing. The deeper difference is that this requires the shell to run in character by character TTY input mode, also called raw mode. By contrast, ed runs in what Unix usually calls cooked mode, where it reads whole lines from the kernel and the kernel handles things like backspace. All of ed's commands are designed so that they work in this line focused way (including being terminated by the end of the line), and as a whole ed's interface makes this whole line input approach natural. In fact I think ed makes it so natural that it's hard to think of things as being any other way. Ed was designed for line at a time input, not just to not be screen oriented.
This input mode difference is not very important today, but in the days of V7 and serial terminals it made a real difference. In cooked mode, V7 ran very little code when you entered each character; almost everything was deferred until it could be processed in bulk by the kernel, and then handed to ed all in a single line which ed could also process all at once. A version of ed that tried to work in raw mode would have been much more resource intensive, even if it still operated on single lines at a time.

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv


Your browser does not support the HTML5 video tag.

May 03 2019

1hr 1min

Play

Rank #8: 310: My New Free NAS

Podcast cover
Read more

OPNsense 19.7.1 is out, ZFS on Linux still has annoying issues with ARC size, Hammer2 is now default, NetBSD audio – an application perspective, new FreeNAS Mini, and more.

Headlines

OPNsense 19.7.1

We do not wish to keep you from enjoying your summer time, but this
is a recommended security update enriched with reliability fixes for the
new 19.7 series. Of special note are performance improvements as well
as a fix for a longstanding NAT before IPsec limitation.

Full patch notes:

  • system: do not create automatic copies of existing gateways
  • system: do not translate empty tunables descriptions
  • system: remove unwanted form action tags
  • system: do not include Syslog-ng in rc.freebsd handler
  • system: fix manual system log stop/start/restart
  • system: scoped IPv6 "%" could confuse mwexecf(), use plain mwexec() instead
  • system: allow curl-based downloads to use both trusted and local authorities
  • system: fix group privilege print and correctly redirect after edit
  • system: use cached address list in referrer check
  • system: fix Syslog-ng search stats
  • firewall: HTML-escape dynamic entries to display aliases
  • firewall: display correct IP version in automatic rules
  • firewall: fix a warning while reading empty outbound rules configuration
  • firewall: skip illegal log lines in live log
  • interfaces: performance improvements for configurations with hundreds of interfaces
  • reporting: performance improvements for Python 3 NetFlow aggregator rewrite
  • dhcp: move advanced router advertisement options to correct config section
  • ipsec: replace global array access with function to ensure side-effect free boot
  • ipsec: change DPD action on start to "dpdaction = restart"
  • ipsec: remove already default "dpdaction = none" if not set
  • ipsec: use interface IP address in local ID when doing NAT before IPsec
  • web proxy: fix database reset for Squid 4 by replacing use of ssl_crtd with security_file_certgen
  • plugins: os-acme-client 1.24[1]
  • plugins: os-bind 1.6[2]
  • plugins: os-dnscrypt-proxy 1.5[3]
  • plugins: os-frr now restricts characters BGP prefix-list and route-maps[4]
  • plugins: os-google-cloud-sdk 1.0[5]
  • ports: curl 7.65.3[6]
  • ports: monit 5.26.0[7]
  • ports: openssh 8.0p1[8]
  • ports: php 7.2.20[9]
  • ports: python 3.7.4[10]
  • ports: sqlite 3.29.0[11]
  • ports: squid 4.8[12]

Stay safe and hydrated, Your OPNsense team

ZFS on Linux still has annoying issues with ARC size

One of the frustrating things about operating ZFS on Linux is that the ARC size is critical but ZFS's auto-tuning of it is opaque and apparently prone to malfunctions, where your ARC will mysteriously shrink drastically and then stick there.

Linux's regular filesystem disk cache is very predictable; if you do disk IO, the cache will relentlessly grow to use all of your free memory. This sometimes disconcerts people when free reports that there's very little memory actually free, but at least you're getting value from your RAM. This is so reliable and regular that we generally don't think about 'is my system going to use all of my RAM as a disk cache', because the answer is always 'yes'. (The general filesystem cache is also called the page cache.)

This is unfortunately not the case with the ZFS ARC in ZFS on Linux (and it wasn't necessarily the case even on Solaris). ZFS has both a current size and a 'target size' for the ARC (called 'c' in ZFS statistics). When your system boots this target size starts out as the maximum allowed size for the ARC, but various events afterward can cause it to be reduced (which obviously limits the size of your ARC, since that's its purpose). In practice, this reduction in the target size is both pretty sticky and rather mysterious (as ZFS on Linux doesn't currently expose enough statistics to tell why your ARC target size shrunk in any particular case).

The net effect is that the ZFS ARC is not infrequently quite shy and hesitant about using memory, in stark contrast to Linux's normal filesystem cache. The default maximum ARC size starts out as only half of your RAM (unlike the regular filesystem cache, which will use all of it), and then it shrinks from there, sometimes very significantly, and once shrunk it only recovers slowly (if at all).

News Roundup

Hammer2 is now default

commit a49112761c919d42d405ec10252eb0553662c824 Author: Matthew Dillon Date: Mon Jun 10 17:53:46 2019 -0700 installer - Default to HAMMER2 * Change the installer default from HAMMER1 to HAMMER2. * Adjust the nrelease build to print the location of the image files when it finishes. Summary of changes: nrelease/Makefile | 2 +- usr.sbin/installer/dfuibe_installer/flow.c | 20 ++++++++++---------- 2 files changed, 11 insertions(+), 11 deletions(-) http://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/a49112761c919d42d405ec10252eb0553662c824

NetBSD audio – an application perspective

NetBSD audio – an application perspective ... or, "doing it natively, because we can"

  • audio options for NetBSD in pkgsrc

    • Use NetBSD native audio (sun audio/audioio.h)
    • Or OSS emulation layer: Basically a wrapper around sun audio in the kernel. Incomplete and old version, but works for simple stuff
  • Many many abstraction layers available:

    • OpenAL-Soft
    • alsa-lib (config file required)
    • libao, GStreamer (plugins!)
    • PortAudio, SDL
    • PulseAudio, JACK
    • ... lots more!? some obsolete stuff (esd, nas?)
  • Advantages of using NetBSD audio directly

    • Low latency, low CPU usage: Abstraction layers differ in latency (SDL2 vs ALSA/OpenAL)
    • Query device information: Is /dev/audio1 a USB microphone or another sound card?
    • Avoid bugs from excessive layering
    • Nice API, well documented: [nia note: I had no idea how to write audio code. I read a man page and now I do.]
    • Your code might work on illumos too
  • [nia note: SDL2 seems very sensitive to the blk_ms sysctl being high or low, with other implementations there seems to be a less noticable difference. I don't know why.]

New FreeNAS Mini

Two new FreeNAS Mini systems join the very popular FreeNAS Mini and Mini XL:

FreeNAS Mini XL+: This powerful 10 Bay platform (8x 3.5” and 1x 2.5” hot-swap, 1x 2.5” internal) includes the latest, compact server technology and provides dual 10GbE ports, 8 CPU cores and 32 GB RAM for high performance workgroups. The Mini XL+ scales beyond 100TB and is ideal for very demanding applications, including hosting virtual machines and multimedia editing. Starting at $1499, the Mini XL+ configured with cache SSD and 80 TB capacity is $4299, and consumes about 100 Watts.

FreeNAS Mini E: This cost-effective 4 Bay platform provides the resources required for SOHO use with quad GbE ports and 8 GB of RAM. The Mini E is ideal for file sharing, streaming and transcoding video at 1080p. Starting at $749, the Mini E configured with 8 TB capacity is $999, and consumes about 36 Watts.

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Your browser does not support the HTML5 video tag.

Aug 08 2019

48mins

Play

Rank #9: Episode 267: Absolute FreeBSD | BSD Now 267

Podcast cover
Read more

We have a long interview with fiction and non-fiction author Michael W. Lucas for you this week as well as questions from the audience.

##Headlines
##Interview - Michael W. Lucas - mwlucas@michaelwlucas.com / @mwlauthor

  • BR: [Welcome Back]
  • AJ: What have you been doing since last we talked to you [ed, ssh, and af3e]
  • BR: Tell us more about AF3e
  • AJ: How did the first Absolute FreeBSD come about?
  • BR: Do you have anything special planned for MeetBSD?
  • AJ: What are you working on now? [FM:Jails, Git sync Murder]
  • BR: What are your plans for next year?
  • AJ: How has SEMIBug been going?

Auction at https://mwl.io
Patreon Link:

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Oct 10 2018

1hr 7mins

Play

Rank #10: 313: In-Kernel TLS

Podcast cover
Read more

OpenBSD on 7th gen Thinkpad X1 Carbon, how to install FreeBSD on a MacBook, Kernel portion of in-kernel TLS (KTLS), Boot Environments on DragonflyBSD, Project Trident Updates, vBSDcon schedule, and more.

Headlines

OpenBSD on the Thinkpad X1 Carbon 7th Gen

Another year, another ThinkPad X1 Carbon, this time with a Dolby Atmos sound system and a smaller battery.
The seventh generation X1 Carbon isn't much different than the fifth and sixth generations. I opted for the non-vPro Core i5-8265U, 16Gb of RAM, a 512Gb NVMe SSD, and a matte non-touch WQHD display at ~300 nits. A brighter 500-nit 4k display is available, though early reports indicated it severely impacts battery life.
Gone are the microSD card slot on the back and 1mm of overall thickness (from 15.95mm to 14.95mm), but also 6Whr of battery (down to 51Whr) and a little bit of travel in the keyboard and TrackPoint buttons. I still very much like the feel of both of them, so kudos to Lenovo for not going too far down the Apple route of sacrificing performance and usability just for a thinner profile.
On my fifth generation X1 Carbon, I used a vinyl plotter to cut out stickers to cover the webcam, "X1 Carbon" branding from the bottom of the display, the power button LED, and the "ThinkPad" branding from the lower part of the keyboard deck.

  • See link for the rest of the article

How To Install FreeBSD On A MacBook 1,1 or 2,1

  • FreeBSD Setup For MacBook 1,1 and 2,1

FreeBSD with some additional setup can be installed on a MacBook 1,1 or 2,1. This article covers how to do so with FreeBSD 10-12.

  • Installing

FreeBSD can be installed as the only OS on your MacBook if desired. What you should have is:

  • A Mac OS X 10.4.6-10.7.5 installer. Unofficial versions modified for these MacBooks such as 10.8 also work.
  • A blank CD or DVD to burn the FreeBSD image to. Discs simply work best with these older MacBooks.
  • An ISO file of FreeBSD for x86. The AMD64 ISO does not boot due to the 32 bit EFI of these MacBooks.
  • Burn the ISO file to the blank CD or DVD. Once done, make sure it's in your MacBook and then power off the MacBook. Turn it on, and hold down the c key until the FreeBSD disc boots.

    • See link for the rest of the guide

News Roundup

Patch for review: Kernel portion of in-kernel TLS (KTLS)

One of the projects I have been working on for the past several months in conjunction with several other folks is upstreaming work from Netflix to handle some aspects of Transport Layer Security (TLS) in the kernel. In particular, this lets a web server use sendfile() to send static content on HTTPS connections. There is a lot more detail in the review itself, so I will spare pasting a big wall of text here. However, I have posted the patch to add the kernel-side of KTLS for review at the URL below. KTLS also requires other patches to OpenSSL and nginx, but this review is only for the kernel bits. Patches and reviews for the other bits will follow later.

DragonFly Boot Enviroments

This is a tool inspired by the beadm utility for FreeBSD/Illumos systems that creates and manages ZFS boot environments. This utility in contrast is written from the ground up in C, this should provide better performance, integration, and extensibility than the POSIX sh and awk script it was inspired by. During the time this project has been worked on, beadm has been superseded by bectl on FreeBSD. After hammering out some of the outstanding internal logic issues, I might look at providing a similar interface to the command as bectl.

  • See link for the rest of the details

Project Trident Updates

This is a general package update to the CURRENT release repository based upon TrueOS 19.08.
Legacy boot ISO functional again
This update includes the FreeBSD fixes for the “vesa” graphics driver for legacy-boot systems. The system can once again be installed on legacy-boot systems.

  • PACKAGE CHANGES FROM 19.07-U1

    • New Packages: 154
    • Deleted Packages: 394
    • Updated Packages: 4926
  • 12-U3 Available

This is the third general package update to the STABLE release repository based upon TrueOS 12-Stable.

  • PACKAGE CHANGES FROM STABLE 12-U2
    • New Packages: 105
    • Deleted Packages: 386
    • Updated Packages: 1046

vBSDcon

  • vBSDcon 2019 will return to the Hyatt Regency in Reston, VA on September 5-7 2019. ***

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Your browser does not support the HTML5 video tag.

Aug 29 2019

55mins

Play

Rank #11: 322: Happy Birthday, Unix

Podcast cover
Read more

Unix is 50, Hunting down Ken's PDP-7, OpenBSD and OPNSense have new releases, Clarification on what GhostBSD is, sshuttle - VPN over SSH, and more.

Headlines

Unix is 50

In the summer of 1969 computer scientists Ken Thompson and Dennis Ritchie created the first implementation of Unix with the goal of designing an elegant and economical operating system for a little-used PDP-7 minicomputer at Bell Labs. That modest project, however, would have a far-reaching legacy. Unix made large-scale networking of diverse computing systems — and the Internet — practical. The Unix team went on to develop the C language, which brought an unprecedented combination of efficiency and expressiveness to programming. Both made computing more "portable". Today, Linux, the most popular descendent of Unix, powers the vast majority of servers, and elements of Unix and Linux are found in most mobile devices. Meanwhile C++ remains one of the most widely used programming languages today. Unix may be a half-century old but its influence is only growing.

Hunting down Ken's PDP-7: video footage found

In my prior blog post, I traced Ken's scrounged PDP-7 to SN 34. In this post I'll show that we have actual video footage of that PDP-7 due to an old film from Bell Labs. this gives us almost a minute of footage of the PDP-7 Ken later used to create Unix.

News Roundup

OpenBSD 6.6 Released

OPNsense 19.7.5 released

Hello friends and followers, Lots of plugin and ports updates this time with a few minor improvements in all core areas. Behind the scenes we are starting to migrate the base system to version

12.1 which is supposed to hit the next 20.1 release. Stay tuned for more infos in the next month or so.

Here are the full patch notes:

  • system: show all swap partitions in system information widget
  • system: flatten services_get() in preparation for removal
  • system: pin Syslog-ng version to specific package name
  • system: fix LDAP/StartTLS with user import page
  • system: fix a PHP warning on authentication server page
  • system: replace most subprocess.call use
  • interfaces: fix devd handling of carp devices (contributed by stumbaumr)
  • firewall: improve firewall rules inline toggles
  • firewall: only allow TCP flags on TCP protocol
  • firewall: simplify help text for direction setting
  • firewall: make protocol log summary case insensitive
  • reporting: ignore malformed flow records
  • captive portal: fix type mismatch for timeout read
  • dhcp: add note for static lease limitation with lease registration (contributed by Northguy)
  • ipsec: add margintime and rekeyfuzz options
  • ipsec: clear $dpdline correctly if not set
  • ui: fix tokenizer reorder on multiple saves
  • plugins: os-acme-client 1.26[1]
  • plugins: os-bind will reload bind on record change (contributed by blablup)
  • plugins: os-etpro-telemetry minor subprocess.call replacement
  • plugins: os-freeradius 1.9.4[2]
  • plugins: os-frr 1.12[3]
  • plugins: os-haproxy 2.19[4]
  • plugins: os-mailtrail 1.2[5]
  • plugins: os-postfix 1.11[6]
  • plugins: os-rspamd 1.8[7]
  • plugins: os-sunnyvalley LibreSSL support (contributed by Sunny Valley Networks)
  • plugins: os-telegraf 1.7.6[8]
  • plugins: os-theme-cicada 1.21 (contributed by Team Rebellion)
  • plugins: os-theme-tukan 1.21 (contributed by Team Rebellion)
  • plugins: os-tinc minor subprocess.call replacement
  • plugins: os-tor 1.8 adds dormant mode disable option (contributed by Fabian Franz)
  • plugins: os-virtualbox 1.0 (contributed by andrewhotlab)

Dealing with the misunderstandings of what is GhostBSD

Since the release of 19.09, I have seen a lot of misunderstandings on what is GhostBSD and the future of GhostBSD. GhostBSD is based on TrueOS with FreeBSD 12 STABLE with our twist to it. We are still continuing to use TrueOS for OpenRC, and the new package's system for the base system that is built from ports. GhostBSD is becoming a slow-moving rolling release base on the latest TrueOS with FreeBSD 12 STABLE. When FreeBSD 13 STABLE gets released, GhostBSD will be upgraded to TrueOS with FreeBSD 13 STABLE.

Our official desktop is MATE, which means that the leading developer of GhostBSD does not officially support XFCE. Community releases are maintained by the community and for the community. GhostBSD project will provide help to build and to host the community release. If anyone wants to have a particular desktop supported, it is up to the community. Sure I will help where I can, answer questions and guide new community members that contribute to community release.

There is some effort going on for Plasma5 desktop. If anyone is interested in helping with XFCE and Plasma5 or in creating another community release, you are well come to contribute. Also, Contribution to the GhostBSD base system, to ports and new ports, and in house software are welcome. We are mostly active on Telegram https://t.me/ghostbsd, but you can also reach us on the forum.

SHUTTLE – VPN over SSH | VPN Alternative

Looking for a lightweight VPN client, but are not ready to spend a monthly recurring amount on a VPN? VPNs can be expensive depending upon the quality of service and amount of privacy you want. A good VPN plan can easily set you back by 10$ a month and even that doesn’t guarantee your privacy. There is no way to be sure whether the VPN is storing your confidential information and traffic logs or not. sshuttle is the answer to your problem it provides VPN over ssh and in this article we’re going to explore this cheap yet powerful alternative to the expensive VPNs. By using open source tools you can control your own privacy.

  • VPN over SSH – sshuttle

sshuttle is an awesome program that allows you to create a VPN connection from your local machine to any remote server that you have ssh access on. The tunnel established over the ssh connection can then be used to route all your traffic from client machine through the remote machine including all the dns traffic. In the bare bones sshuttle is just a proxy server which runs on the client machine and forwards all the traffic to a ssh tunnel. Since its open source it holds quite a lot of major advantages over traditional VPN.

OpenSSH 8.1 Released

  • Security

    • ssh(1), sshd(8), ssh-add(1), ssh-keygen(1): an exploitable integer overflow bug was found in the private key parsing code for the XMSS key type. This key type is still experimental and support for it is not compiled by default. No user-facing autoconf option exists in portable OpenSSH to enable it. This bug was found by Adam Zabrocki and reported via SecuriTeam's SSD program.
    • ssh(1), sshd(8), ssh-agent(1): add protection for private keys at rest in RAM against speculation and memory side-channel attacks like Spectre, Meltdown and Rambleed. This release encrypts private keys when they are not in use with a symmetric key that is derived from a relatively large "prekey" consisting of random data (currently 16KB).
  • This release includes a number of changes that may affect existing configurations:

    • ssh-keygen(1): when acting as a CA and signing certificates with an RSA key, default to using the rsa-sha2-512 signature algorithm. Certificates signed by RSA keys will therefore be incompatible with OpenSSH versions prior to 7.2 unless the default is overridden (using "ssh-keygen -t ssh-rsa -s ...").
  • New Features

    • ssh(1): Allow %n to be expanded in ProxyCommand strings
    • ssh(1), sshd(8): Allow prepending a list of algorithms to the default set by starting the list with the '' character, E.g. "HostKeyAlgorithms ssh-ed25519"
    • ssh-keygen(1): add an experimental lightweight signature and verification ability. Signatures may be made using regular ssh keys held on disk or stored in a ssh-agent and verified against an authorized_keys-like list of allowed keys. Signatures embed a namespace that prevents confusion and attacks between different usage domains (e.g. files vs email).
    • ssh-keygen(1): print key comment when extracting public key from a private key.
    • ssh-keygen(1): accept the verbose flag when searching for host keys in known hosts (i.e. "ssh-keygen -vF host") to print the matching host's random-art signature too.
    • All: support PKCS8 as an optional format for storage of private keys to disk. The OpenSSH native key format remains the default, but PKCS8 is a superior format to PEM if interoperability with non-OpenSSH software is required, as it may use a less insecure key derivation function than PEM's.

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

Oct 31 2019

1hr 7mins

Play

Rank #12: 337: Kubernetes on bhyve

Podcast cover
Read more

Happinesses and stresses of full-time FOSS work, building a FreeBSD fileserver, Kubernetes on FreeBSD bhyve, NetBSD 9 RC1 available, OPNSense 20.1 is here, HardenedBSD’s idealistic future, and more.

Headlines

The happinesses and stresses of full-time FOSS work

In the past few days, several free software maintainers have come out to discuss the stresses of their work. Though the timing was suggestive, my article last week on the philosophy of project governance was, at best, only tangentially related to this topic - I had been working on that article for a while. I do have some thoughts that I’d like to share about what kind of stresses I’ve dealt with as a FOSS maintainer, and how I’ve managed (or often mismanaged) it.

February will mark one year that I’ve been working on self-directed free software projects full-time. I was planning on writing an optimistic retrospective article around this time, but given the current mood of the ecosystem I think it would be better to be realistic. In this stage of my career, I now feel at once happier, busier, more fulfilled, more engaged, more stressed, and more depressed than I have at any other point in my life.

The good parts are numerous. I’m able to work on my life’s passions, and my projects are in the best shape they’ve ever been thanks to the attention I’m able to pour into them. I’ve also been able to do more thoughtful, careful work; with the extra time I’ve been able to make my software more robust and reliable than it’s ever been. The variety of projects I can invest my time into has also increased substantially, with what was once relegated to minor curiosities now receiving a similar amount of attention as my larger projects were receiving in my spare time before. I can work from anywhere in the world, at any time, not worrying about when to take time off and when to put my head down and crank out a lot of code.

The frustrations are numerous, as well. I often feel like I’ve bit off more than I can chew. This has been the default state of affairs for me for a long time; I’m often neglecting half of my projects in order to obtain progress by leaps and bounds in just a few. Working on FOSS full-time has cast this model’s disadvantages into greater relief, as I focus on a greater breadth of projects and spend more time on them.

Building a FreeBSD File Server

Recently at my job, I was faced with a task to develop a file server explicitly suited for the requirements of the company. Needless to say, any configuration of a kind depends on what the infrastructure needs. So, drawing from my personal experience and numerous materials on the web, I came up with the combination FreeBSD+SAMBA+AD as the most appropriate. It appears to be a perfect choice for this environment, and harmonic addition to the existing network configuration since FreeBSD + SAMBA + AD enables admins with the broad range of possibilities for access control. However, as nothing is perfect, this configuration isn’t the best choice if your priority is data protection because it won’t be able to reach the necessary levels of reliability and fault tolerance without outside improvements.

Now, since we’ve established that, let’s move on to the next point. This article’s describing the process of building a test environment while concentrating primarily on the details of the configuration. As the author, though, I must say I’m in no way suggesting that this is the only way! The following configuration will be presented in its initial stage, with the minimum requirements necessary to get the job done, and its purpose in one specific situation only. Here, look at this as a useful strategy to solve similar tasks. Well, let’s get started!

Report from the first Hamilton BSD Users Group Meeting

February 11th was the first meeting of this new user group, founded by John Young and myself

11 people attended, and a lot of good discussions were had

One of the attendees already owns a domain that fits well for the group, so we will be getting that setup over the next few weeks, as well as the twitter account, and other organization stuff.

Special thanks to the illumos users who drove in from Buffalo to attend, although they may have actually had a shorter drive than a few of the other attendees.

The next meeting is scheduled again for the 2nd Tuesday of the month, March 10th.

We are still discussing if we should meet at a restaurant again, or try to get a space at the local college or innovation hub where we can have a projector etc.

News Roundup

Kubernetes on FreeBSD Bhyve

There are quite a few solutions for container orchestration, but the most popular (or the most famous and highly advertised, is probably, a Kubernetes) Since I plan to conduct many experiments with installing and configuring k8s, I need a laboratory in which I can quickly and easily deploy a cluster in any quantities for myself. In my work and everyday life I use two OS very tightly - Linux and FreeBSD OS. Kubernetes and docker are Linux-centric projects, and at first glance, you should not expect any useful participation and help from FreeBSD here. As the saying goes, an elephant can be made out of a fly, but it will no longer fly. However, two tempting things come to mind - this is very good integration and work in the FreeBSD ZFS file system, from which it would be nice to use the snapshot mechanism, COW and reliability. And the second is the bhyve hypervisor, because we still need the docker and k8s loader in the form of the Linux kernel. Thus, we need to connect a certain number of actions in various ways, most of which are related to starting and pre-configuring virtual machines. This is typical of both a Linux-based server and FreeBSD. What exactly will work under the hood to run virtual machines does not play a big role. And if so - let's take a FreeBSD here!

NetBSD 9 RC1 Available

We hope this will lead to the best NetBSD release ever (only to be topped by NetBSD 10 next year).

  • Here are a few highlights of the new release:

    • Support for Arm AArch64 (64-bit Armv8-A) machines, including "Arm ServerReady" compliant machines (SBBR+SBSA)
    • Enhanced hardware support for Armv7-A
    • Updated GPU drivers (e.g. support for Intel Kabylake)
    • Enhanced virtualization support
    • Support for hardware-accelerated virtualization (NVMM)
    • Support for Performance Monitoring Counters
    • Support for Kernel ASLR
    • Support several kernel sanitizers (KLEAK, KASAN, KUBSAN)
    • Support for userland sanitizers
    • Audit of the network stack
    • Many improvements in NPF
    • Updated ZFS
    • Reworked error handling and NCQ support in the SATA subsystem
    • Support a common framework for USB Ethernet drivers (usbnet)
  • You can download binaries of NetBSD 9.0_RC1 from our Fastly-provided CDN: https://cdn.netbsd.org/pub/NetBSD/NetBSD-9.0_RC1/

OPNsense 20.1 Keen Kingfisher released

For over 5 years now, OPNsense is driving innovation through modularising and hardening the open source firewall, with simple and reliable firmware upgrades, multi-language support, HardenedBSD security, fast adoption of upstream software updates as well as clear and stable 2-Clause BSD licensing.

20.1, nicknamed "Keen Kingfisher", is a subtle improvement on sustainable firewall experience. This release adds VXLAN and additional loopback device support, IPsec public key authentication and elliptic curve TLS certificate creation amongst others. Third party software has been updated to their latest versions. The logging frontend was rewritten for MVC with seamless API support. On the far side the documentation increased in quality as well as quantity and now presents itself in a familiar menu layout.

Idealistic Future for HardenedBSD

Over the past month, we purchased and deployed the new 13-CURRENT/amd64 package building server. We published our first 13-CURRENT/amd64 production package build using that server. We then rebuilt the old package building server to act as the 12-STABLE/amd64 package building server. This post signifies a very important milestone: we have now fully recovered from last year's death of our infrastructure. Our 12-STABLE/amd64 repo, previously out-of-date by many months, is now fully up-to-date!

HardenedBSD is in a very unique position to provide innovative solutions to at-risk and underprivileged populations. As such, we are making human rights endeavors a defining area of focus. Our infrastructure will integrate various privacy and anonymity enhancing technologies and techniques to protect lives. Our operating system's security posture will increase, especially with our focus on exploit mitigations.

Navigating the intersection between human rights and information security directly impacts lives. HardenedBSD's 2020 mission and focus is to deliver an entire hardened ecosystem that is unfriendly towards those who would oppress or censor their people. This includes a subtle shift in priorities to match this new mission and focus. While we implement exploit mitigations and further harden the ecosystem, we will seek out opportunities to contribute a tangible and unique impact on human rights issues. Providing Tor Onion Services for our core infrastructure is the first step in likely many to come towards securely helping those in need.

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

Feb 13 2020

1hr 19mins

Play

Rank #13: Episode 243: Understanding The Scheduler | BSD Now 243

Podcast cover
Read more

OpenBSD 6.3 and DragonflyBSD 5.2 are released, bug fix for disappearing files in OpenZFS on Linux (and only Linux), understanding the FreeBSD CPU scheduler, NetBSD on RPI3, thoughts on being a committer for 20 years, and 5 reasons to use FreeBSD in 2018.

Headlines

OpenBSD 6.3 released

  • Punctual as ever, OpenBSD 6.3 has been releases with the following features/changes:

    Improved HW support, including:
    SMP support on OpenBSD/arm64 platforms
    vmm/vmd improvements:
    IEEE 802.11 wireless stack improvements
    Generic network stack improvements
    Installer improvements
    Routing daemons and other userland network improvements
    Security improvements
    dhclient(8) improvements
    Assorted improvements
    OpenSMTPD 6.0.4
    OpenSSH 7.7
    LibreSSL 2.7.2

DragonFlyBSD 5.2 released

Big-ticket items
Meltdown and Spectre mitigation support
Meltdown isolation and spectre mitigation support added. Meltdown mitigation is automatically enabled for all Intel cpus. Spectre mitigation must be enabled manually via sysctl if desired, using sysctls machdep.spectremitigation and machdep.meltdownmitigation.
HAMMER2
H2 has received a very large number of bug fixes and performance improvements. We can now recommend H2 as the default root filesystem in non-clustered mode.
Clustered support is not yet available.
ipfw Updates
Implement state based "redirect", i.e. without using libalias.
ipfw now supports all possible ICMP types.
Fix ICMPMAXTYPE assumptions (now 40 as of this release).
Improved graphics support
The drm/i915 kernel driver has been updated to support Intel Coffeelake GPUs
Add 24-bit pixel format support to the EFI frame buffer code.
Significantly improve fbio support for the "scfb" XOrg driver. This allows EFI frame buffers to be used by X in situations where we do not otherwise support the GPU.
Partly implement the FBIO
BLANK ioctl for display powersaving.
Syscons waits for drm modesetting at appropriate places, avoiding races.
+ For more details, check out the “All changes since DragonFly 5.0” section.

ZFS on Linux bug causes files to disappear

  • A bug in ZoL 0.7.7 caused 0.7.8 to be released just 3 days after the release
  • The bug only impacts Linux, the change that caused the problem was not upstreamed yet, so does not impact ZFS on illumos, FreeBSD, OS X, or Windows
  • The bug can cause files being copied into a directory to not be properly linked to the directory, so they will no longer be listed in the contents of the directory
  • ZoL developers are working on a tool to allow you to recover the data, since no data was actually lost, the files were just not properly registered as part of the directory
  • The bug was introduced in a commit made in February, that attempted to improve performance of datasets created with the case insensitivity option. In an effort to improve performance, they introduced a limit to cap to give up (return ENOSPC) if growing the directory ZAP failed twice.
  • The ZAP is the key-value pair data structure that contains metadata for a directory, including a hash table of the files that are in a directory. When a directory has a large number of files, the ZAP is converted to a FatZAP, and additional space may need to be allocated as additional files are added.

    Commit cc63068 caused ENOSPC error when copy a large amount of files between two directories. The reason is that the patch limits zap leaf expansion to 2 retries, and return ENOSPC when failed.
  • Finding the root cause of this issue was somewhat hampered by the fact that many people were not able to reproduce the issue. It turns out this was caused by an entirely unrelated change to GNU coreutils.
  • On later versions of GNU Coreutils, the files were returned in a sorted order, resulting in them hitting different buckets in the hash table, and not tripping the retry limit
  • Tools like rsync were unaffected, because they always sort the files before copying
  • If you did not see any ENOSPC errors, you were likely not impacted
    The intent for limiting retries is to prevent pointlessly growing table to max size when adding a block full of entries with same name in different case in mixed mode. However, it turns out we cannot use any limit on the retry. When we copy files from one directory in readdir order, we are copying in hash order, one leaf block at a time. Which means that if the leaf block in source directory has expanded 6 times, and you copy those entries in that block, by the time you need to expand the leaf in destination directory, you need to expand it 6 times in one go. So any limit on the retry will result in error where it shouldn't.
  • Recommendations for Users from Ryan Yao:
    The regression makes it so that creating a new file could fail with ENOSPC after which files created in that directory could become orphaned. Existing files seem okay, but I have yet to confirm that myself and I cannot speak for what others know. It is incredibly difficult to reproduce on systems running coreutils 8.23 or later. So far, reports have only come from people using coreutils 8.22 or older. The directory size actually gets incremented for each orphaned file, which makes it wrong after orphan files happen.
    We will likely have some way to recover the orphaned files (like ext4’s lost+found) and fix the directory sizes in the very near future. Snapshots of the damaged datasets are problematic though. Until we have a subcommand to fix it (not including the snapshots, which we would have to list), the damage can be removed from a system that has it either by rolling back to a snapshot before it happened or creating a new dataset with 0.7.6 (or another release other than 0.7.7), moving everything to the new dataset and destroying the old. That will restore things to pristine condition.
    It should also be possible to check for pools that are affected, but I have yet to finish my analysis to be certain that no false negatives occur when checking, so I will avoid saying how for now.
  • Writes to existing files cannot trigger this bug, only adding new files to a directory in bulk

News Roundup

des@’s thoughts on being a FreeBSD committer for 20 years

Yesterday was the twentieth anniversary of my FreeBSD commit bit, and tomorrow will be the twentieth anniversary of my first commit. I figured I’d split the difference and write a few words about it today.

My level of engagement with the FreeBSD project has varied greatly over the twenty years I’ve been a committer. There have been times when I worked on it full-time, and times when I did not touch it for months. The last few years, health issues and life events have consumed my time and sapped my energy, and my contributions have come in bursts. Commit statistics do not tell the whole story, though: even when not working on FreeBSD directly, I have worked on side projects which, like OpenPAM, may one day find their way into FreeBSD.

My contributions have not been limited to code. I was the project’s first Bugmeister; I’ve served on the Security Team for a long time, and have been both Security Officer and Deputy Security Officer; I managed the last four Core Team elections and am doing so again this year.

In return, the project has taught me much about programming and software engineering. It taught me code hygiene and the importance of clarity over cleverness; it taught me the ins and outs of revision control; it taught me the importance of good documentation, and how to write it; and it taught me good release engineering practices.

Last but not least, it has provided me with the opportunity to work with some of the best people in the field. I have the privilege today to count several of them among my friends.

For better or worse, the FreeBSD project has shaped my career and my life. It set me on the path to information security in general and IAA in particular, and opened many a door for me. I would not be where I am now without it.

I won’t pretend to be able to tell the future. I don’t know how long I will remain active in the FreeBSD project and community. It could be another twenty years; or it could be ten, or five, or less. All I know is that FreeBSD and I still have things to teach each other, and I don’t intend to call it quits any time soon.

iXsystems unveils new TrueNAS M-Series Unified Storage Line

San Jose, Calif., April 10, 2018 — iXsystems, the leader in Enterprise Open Source servers and software-defined storage, announced the TrueNAS M40 and M50 as the newest high-performance models in its hybrid, unified storage product line. The TrueNAS M-Series harnesses NVMe and NVDIMM to bring all-flash array performance to the award-winning TrueNAS hybrid arrays. It also includes the Intel® Xeon® Scalable Family of Processors and supports up to 100GbE and 32Gb Fibre Channel networking. Sitting between the all-flash TrueNAS Z50 and the hybrid TrueNAS X-Series in the product line, the TrueNAS M-Series delivers up to 10 Petabytes of highly-available and flash-powered network attached storage and rounds out a comprehensive product set that has a capacity and performance option for every storage budget.

  • Designed for On-Premises & Enterprise Cloud Environments

As a unified file, block, and object sharing solution, TrueNAS can meet the needs of file serving, backup, virtualization, media production, and private cloud users thanks to its support for the SMB, NFS, AFP, iSCSI, Fibre Channel, and S3 protocols.

At the heart of the TrueNAS M-Series is a custom 4U, dual-controller head unit that supports up to 24 3.5” drives and comes in two models, the M40 and M50, for maximum flexibility and scalability. The TrueNAS M40 uses NVDIMMs for write cache, SSDs for read cache, and up to two external 60-bay expansion shelves that unlock up to 2PB in capacity. The TrueNAS M50 uses NVDIMMs for write caching, NVMe drives for read caching, and up to twelve external 60-bay expansion shelves to scale upwards of 10PB. The dual-controller design provides high-availability failover and non-disruptive upgrades for mission-critical enterprise environments.

By design, the TrueNAS M-Series unleashes cutting-edge persistent memory technology for demanding performance and capacity workloads, enabling businesses to accelerate enterprise applications and deploy enterprise private clouds that are twice the capacity of previous TrueNAS models. It also supports replication to the Amazon S3, BackBlaze B2, Google Cloud, and Microsoft Azure cloud platforms and can deliver an object store using the ubiquitous S3 object storage protocol at a fraction of the cost of the public cloud.

  • Fast

As a true enterprise storage platform, the TrueNAS M50 supports very demanding performance workloads with up to four active 100GbE ports, 3TB of RAM, 32GB of NVDIMM write cache and up to 15TB of NVMe flash read cache. The TrueNAS M40 and M50 include up to 24/7 and global next-business-day support, putting IT at ease. The modular and tool-less design of the M-Series allows for easy, non-disruptive servicing and upgrading by end-users and support technicians for guaranteed uptime. TrueNAS has US-Based support provided by the engineering team that developed it, offering the rapid response that every enterprise needs.

  • Award-Winning TrueNAS Features

    • Enterprise: Perfectly suited for private clouds and enterprise workloads such as file sharing, backups, M&E, surveillance, and hosting virtual machines.
    • Unified: Utilizes SMB, AFP, NFS for file storage, iSCSI, Fibre Channel and OpenStack Cinder for block storage, and S3-compatible APIs for object storage. Supports every common operating system, hypervisor, and application.
    • Economical: Deploy an enterprise private cloud and reduce storage TCO by 70% over AWS with built-in enterprise-class features such as in-line compression, deduplication, clones, and thin-provisioning.
    • Safe: The OpenZFS file system ensures data integrity with best-in-class replication and snapshotting. Customers can replicate data to the rest of the iXsystems storage lineup and to the public cloud.
    • Reliable: High Availability option with dual hot-swappable controllers for continuous data availability and 99.999% uptime.
    • Familiar: Provision and manage storage with the same simple and powerful WebUI and REST APIs used in all iXsystems storage products, as well as iXsystems’ FreeNAS Software.
    • Certified: TrueNAS has passed the Citrix Ready, VMware Ready, and Veeam Ready certifications, reducing the risk of deploying a virtualized infrastructure.
    • Open: By using industry-standard sharing protocols, the OpenZFS Open Source enterprise file system and FreeNAS, the world’s #1 Open Source storage operating system (and also engineered by iXsystems), TrueNAS is the most open enterprise storage solution on the market.
  • Availability

The TrueNAS M40 and M50 will be generally available in April 2018 through the iXsystems global channel partner network. The TrueNAS M-Series starts at under $20,000 USD and can be easily expanded using a linear “per terabyte” pricing model. With typical compression, a Petabtye can be stored for under $100,000 USD. TrueNAS comes with an all-inclusive software suite that provides NFS, Windows SMB, iSCSI, snapshots, clones and replication.

Understanding and tuning the FreeBSD Scheduler

```
Occasionally I noticed that the system would not quickly process the
tasks i need done, but instead prefer other, longrunning tasks. I
figured it must be related to the scheduler, and decided it hates me.

A closer look shows the behaviour as follows (single CPU):

Lets run an I/O-active task, e.g, postgres VACUUM that would
continuously read from big files (while doing compute as well [1]):

pool alloc free read write read write
cache - - - - - -
ada1s4 7.08G 10.9G 1.58K 0 12.9M 0

Now start an endless loop:

while true; do :; done

And the effect is:

pool alloc free read write read write
cache - - - - - -
ada1s4 7.08G 10.9G 9 0 76.8K 0

The VACUUM gets almost stuck! This figures with WCPU in "top":

PID USERNAME PRI NICE SIZE RES STATE TIME WCPU COMMAND
85583 root 99 0 7044K 1944K RUN 1:06 92.21% bash
53005 pgsql 52 0 620M 91856K RUN 5:47 0.50% postgres

Hacking on kern.sched.quantum makes it quite a bit better:

sysctl kern.sched.quantum=1

kern.sched.quantum: 94488 -> 7874

pool alloc free read write read write
cache - - - - - -
ada1s4 7.08G 10.9G 395 0 3.12M 0

PID USERNAME PRI NICE SIZE RES STATE TIME WCPU COMMAND
85583 root 94 0 7044K 1944K RUN 4:13 70.80% bash
53005 pgsql 52 0 276M 91856K RUN 5:52 11.83% postgres

Now, as usual, the "root-cause" questions arise: What exactly does
this "quantum"? Is this solution a workaround, i.e. actually something
else is wrong, and has it tradeoff in other situations? Or otherwise,
why is such a default value chosen, which appears to be ill-deceived?

The docs for the quantum parameter are a bit unsatisfying - they say
its the max num of ticks a process gets - and what happens when
they're exhausted? If by default the endless loop is actually allowed
to continue running for 94k ticks (or 94ms, more likely) uninterrupted,
then that explains the perceived behaviour - buts thats certainly not
what a scheduler should do when other procs are ready to run.

11.1-RELEASE-p7, kern.hz=200. Switching tickless mode on or off does
not influence the matter. Starting the endless loop with "nice" does
not influence the matter.

[1]
A pure-I/O job without compute load, like "dd", does not show
this behaviour. Also, when other tasks are running, the unjust
behaviour is not so stongly pronounced.
```

aarch64 support added

I have committed about adding initial support for aarch64.

  • booting log on RaspberryPI3:

```
boot NetBSD/evbarm (aarch64)
Drop to EL1...OK
Creating VA=PA tables
Creating KSEG tables
Creating KVA=PA tables
Creating devmap tables
MMU Enable...OK
VSTART = ffffffc000001ff4
FDT<3ab46000> devmap cpufunc bootstrap consinit ok
uboot: args 0x3ab46000, 0, 0, 0

NetBSD/evbarm (fdt) booting ...
FDT /memory [0] @ 0x0 size 0x3b000000
MEM: add 0-3b000000
MEM: res 0-1000
MEM: res 3ab46000-3ab4a000
Usable memory:
1000 - 3ab45fff
3ab4a000 - 3affffff
initarm: kernel phys start 1000000 end 17bd000
MEM: res 1000000-17bd000
bootargs: root=axe0
1000 - ffffff
17bd000 - 3ab45fff
3ab4a000 - 3affffff
------------------------------------------
kern_vtopdiff = 0xffffffbfff000000
physical_start = 0x0000000000001000
kernel_start_phys = 0x0000000001000000
kernel_end_phys = 0x00000000017bd000
physical_end = 0x000000003ab45000
VM_MIN_KERNEL_ADDRESS = 0xffffffc000000000
kernel_start_l2 = 0xffffffc000000000
kernel_start = 0xffffffc000000000
kernel_end = 0xffffffc0007bd000
kernel_end_l2 = 0xffffffc000800000
(kernel va area)
(devmap va area)
VM_MAX_KERNEL_ADDRESS = 0xffffffffffe00000
------------------------------------------
Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005,
2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017,
2018 The NetBSD Foundation, Inc. All rights reserved.
Copyright (c) 1982, 1986, 1989, 1991, 1993
The Regents of the University of California. All rights reserved.

NetBSD 8.99.14 (RPI64) #11: Fri Mar 30 12:34:19 JST 2018
ryo@moveq:/usr/home/ryo/tmp/netbsd-src-ryo-wip/sys/arch/evbarm/compile/RPI64
total memory = 936 MB
avail memory = 877 MB

Starting local daemons:.
Updating motd.
Starting sshd.
Starting inetd.
Starting cron.
The following components reported failures:
/etc/rc.d/swap2
See /var/run/rc.log for more information.
Fri Mar 30 12:35:31 JST 2018

NetBSD/evbarm (rpi3) (console)

login: root
Last login: Fri Mar 30 12:30:24 2018 on console

rpi3# uname -ap
NetBSD rpi3 8.99.14 NetBSD 8.99.14 (RPI64) #11: Fri Mar 30 12:34:19 JST 2018 ryo@moveq:/usr/home/ryo/tmp/netbsd-src-ryo-wip/sys/arch/evbarm/compile/RPI64 evbarm aarch64
rpi3#

```

Now, multiuser mode works stably on fdt based boards (RPI3,SUNXI,TEGRA). But there are still some problems, more time is required for release. also SMP is not yet. See sys/arch/aarch64/aarch64/TODO for more detail. Especially the problems around TLS of rtld, and C++ stack unwindings are too difficult for me to solve, I give up and need someone's help (^o^)/ Since C++ doesn't work, ATF also doesn't work. If the ATF works, it will clarify more issues.

sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. One evbarm/conf/GENERIC64 kernel binary supports all fdt (bcm2837,sunxi,tegra) based boards. While on 32bit, sys/arch/evbarm/conf/GENERIC will support all fdt based boards...but doesn't work yet. (WIP)

My deepest appreciation goes to Tohru Nishimura (nisimura@) whose writes vector handlers, context switchings, and so on. and his comments and suggestions were innumerably valuable. I would also like to thank Nick Hudson (skrll@) and Jared McNeill (jmcneill@) whose added support FDT and integrated into evbarm. Finally, I would like to thank Matt Thomas (matt@) whose commited aarch64
toolchains and preliminary support for aarch64.

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Apr 25 2018

1hr 25mins

Play

Rank #14: Episode 246: Properly Coordinated Disclosure | BSD Now 246

Podcast cover
Read more

How Intel docs were misinterpreted by almost any OS, a look at the mininet SDN emulator, do’s and don’ts for FreeBSD, OpenBSD community going gold, ed mastery is a must read, and the distributed object store minio on FreeBSD.

Headlines

Intel documentation flaw sees instruction misimplemented in almost every OS

A statement in the System Programming Guide of the Intel 64 and IA-32 Architectures Software Developer's Manual (SDM) was mishandled in the development of some or all operating-system kernels, resulting in unexpected behavior for #DB exceptions that are deferred by MOV SS or POP SS, as demonstrated by (for example) privilege escalation in Windows, macOS, some Xen configurations, or FreeBSD, or a Linux kernel crash. OS kernels may not expect this order of events and may therefore experience unexpected behavior when it occurs. + A detailed white paper describes this behavior here + FreeBSD Commit Thank you to the MSRC Incident Response Team, and in particular Greg Lenti and Nate Warfield, for coordinating the response to this issue across multiple vendors. Thanks to Computer Recycling at The Working Center of Kitchener for making hardware available to allow us to test the patch on additional CPU families. + FreeBSD Security Advisory + DragonFlyBSD Post + NetBSD does not support debug register and so is not affected. + OpenBSD also appears to not be affected, “We are not aware of further vendor information regarding this vulnerability.” + IllumOS Not Impacted

Guest Post – A Look at SDN Emulator Mininet

  • A guest post on the FreeBSD Foundation’s blog by developer Ayaka Koshibe
    At this year’s AsiaBSDCon, I presented a talk about a SDN network emulator called Mininet, and my ongoing work to make it more portable. That presentation was focused on the OpenBSD version of the port, and I breezed past the detail that I also had a version or Mininet working on FreeBSD. Because I was given the opportunity, I’d like to share a bit about the FreeBSD version of Mininet. It will not only be about what Mininet is and why it might be interesting, but also a recounting of my experience as a user making a first-time attempt at porting an application to FreeBSD. Mininet started off as a tool used by academic researchers to emulate OpenFlow networks when they didn’t have convenient access to actual networks. Because of its history, Mininet became associated strongly with networks that use OpenFlow for their control channels. But, it has also become fairly popular among developers working in, and among several universities for research and teaching about, SDN (Software Defined Networking) I began using Mininet as an intern at my university’s network research lab. I was using FreeBSD by that time, and wasn’t too happy to learn that Mininet wouldn’t work on anything but Linux. I gradually got tired of having to run a Linux VM just to use Mininet, and one day it clicked in my mind that I can actually try porting it to FreeBSD. Mininet creates a topology using the resource virtualization features that Linux has. Specifically, nodes are bash processes running in network namespaces, and the nodes are interconnected using veth virtual Ethernet links. Switches and controllers are just nodes whose shells have run the right commands to configure a software switch or start a controller application. Mininet can therefore be viewed as a series of Python libraries that run the system commands necessary to create network namespaces and veth interfaces, assemble a specified topology, and coordinate how user commands aimed at nodes (since they are just shells) are run. Coming back to the port, I chose to use vnet jails to replace the network namespaces, and epair(4) links to replace the veth links. For the SDN functionality, I needed at least one switch and controller that can be run on FreeBSD. I chose OpenvSwitch(OVS) for the switch, since it was available in ports and is well-known by the SDN world, and Ryu for the controller since it’s being actively developed and used and supports more recent versions of OpenFlow. I have discussed the possibility of upstreaming my work. Although they were excited about it, I was asked about a script for creating VMs with Mininet preinstalled, and continuous integration support for my fork of the repository. I started taking a look at the release scripts for creating a VM, and after seeing that it would be much easier to use the scripts if I can get Mininet and Ryu added to the ports tree, I also tried a hand at submitting some ports. For CI support, Mininet uses Travis, which unfortunately doesn’t support FreeBSD. For this, I plan to look at a minimalistic CI tool called contbuild, which looks simple enough to get running and is written portably. This is very much a work-in-progress, and one going at a glacial pace. Even though the company that I work for does use Mininet, but doesn’t use FreeBSD, so this is something that I’ve been working on in my free time. Earlier on, it was the learning curve that made progress slow. When I started, I hadn’t done anything more than run FreeBSD on a laptop, and uneventfully build a few applications from the ports tree. Right off the bat, using vnet jails meant learning how to build and run a custom kernel. This was the easy part, as the handbook was clear about how to do this. When I moved from using FreeBSD 10.3 to 11, I found that I can panic my machine by quickly creating and destroying OVS switches and jails. I submitted a bug report, but decided to go one step further and actually try to debug the panic for myself. With the help of a few people well-versed in systems programming and the developer’s handbook, I was able to come up with a fix, and get it accepted. This pretty much brings my porting experiment to the present day, where I’m slowly working out the pieces that I mentioned earlier. In the beginning, I thought that this Mininet port would be a weekend project where I come out knowing thing or two about using vnet jails and with one less VM to run. Instead, it became a crash course in building and debugging kernels and submitting bug reports, patches, and ports. It’d like to mention that I wouldn’t have gotten far at all if it weren’t for the helpful folks, the documentation, and how debuggable FreeBSD is. I enjoy good challenges and learning experiences, and this has definitely been both.
  • Thank you to Ayaka for working to port Mininet to the BSDs, and for sharing her experiences with us.
  • If you want to see the OpenBSD version of the talk, the video from AsiaBSDCon is here, and it will be presented again at BSDCan.


iXsystems
iXsystems LFNW Recap

10 Beginner Do's and Don't for FreeBSD

  • 1) Don't mix ports and binary packages
  • 2) Don't edit 'default' files
  • 3) Don't mess with /etc/crontab
  • 4) Don't mess with /etc/passwd and /etc/groups either!
  • 5) Reconsider the removal of any options from your customized kernel configuration
  • 6) Don't change the root shell to something else
  • 7) Don't use the root user all the time
  • 8) /var/backups is a thing
  • 9) Check system integrity using /etc/mtree
  • 10) What works for me doesn't have to work for you!

News Roundup

OpenBSD Community Goes Gold for 2018!

  • Ken Westerback (krw@ when wearing his developer hat) writes:

``` Monthly paypal donations from the OpenBSD community have made the community the OpenBSD Foundation's first Gold level contributor for 2018!

2018 is the third consecutive year that the community has reached Gold status or better.

These monthly paypal commitments by the community are our most reliable source of funds and thus the most useful for financial planning purposes. We are extremely thankful for the continuing support and hope the community matches their 2017 achievement of Platinum status. Or even their 2016 achievement of Iridium status.

Sign up now for a monthly donation!

Note that Bitcoin contributions have been re-enabled now that our Bitcoin intermediary has re-certified our Canadian paperwork.

https://www.openbsdfoundation.org/donations.html ```

ed(1) mastery is a must read for real unix people

In some circles on the Internet, your choice of text editor is a serious matter.

We've all seen the threads on mailing lits, USENET news groups and web forums about the relative merits of Emacs vs vi, including endless iterations of flame wars, and sometimes even involving lesser known or non-portable editing environments.

And then of course, from the Linux newbies we have seen an endless stream of tweeted graphical 'memes' about the editor vim (aka 'vi Improved') versus the various apparently friendlier-to-some options such as GNU nano. Apparently even the 'improved' version of the classical and ubiquitous vi(1) editor is a challenge even to exit for a significant subset of the younger generation.

Yes, your choice of text editor or editing environment is a serious matter. Mainly because text processing is so fundamental to our interactions with computers.

But for those of us who keep our systems on a real Unix (such as OpenBSD or FreeBSD), there is no real contest. The OpenBSD base system contains several text editors including vi(1) and the almost-emacs mg(1), but ed(1) remains the standard editor.

Now Michael Lucas has written a book to guide the as yet uninitiated to the fundamentals of the original Unix text editor. It is worth keeping in mind that much of Unix and its original standard text editor written back when the standard output and default user interface was more likely than not a printing terminal.

To some of us, reading and following the narrative of Ed Mastery is a trip down memory lane. To others, following along the text will illustrate the horror of the world of pre-graphic computer interfaces. For others again, the fact that ed(1) doesn't use your terminal settings much at all offers hope of fixing things when something or somebody screwed up your system so you don't have a working terminal for that visual editor.

DigitalOcean Digital Ocean Promo Link for BSD Now Listeners

Distributed Object Storage with Minio on FreeBSD

Free and open source distributed object storage server compatible with Amazon S3 v2/v4 API. Offers data protection against hardware failures using erasure code and bitrot detection. Supports highly available distributed setup. Provides confidentiality, integrity and authenticity assurances for encrypted data with negligible performance overhead. Both server side and client side encryption are supported. Below is the image of example Minio setup.

The Minio identifies itself as the ZFS of Cloud Object Storage. This guide will show You how to setup highly available distributed Minio storage on the FreeBSD operating system with ZFS as backend for Minio data. For convenience we will use FreeBSD Jails operating system level virtualization.

  • Setup

The setup will assume that You have 3 datacenters and assumption that you have two datacenters in whose the most of the data must reside and that the third datacenter is used as a ‘quorum/witness’ role. Distributed Minio supports up to 16 nodes/drives total, so we may juggle with that number to balance data between desired datacenters. As we have 16 drives to allocate resources on 3 sites we will use 7 + 7 + 2 approach here. The datacenters where most of the data must reside have 7/16 ratio while the ‘quorum/witness’ datacenter have only 2/16 ratio. Thanks to built in Minio redundancy we may loose (turn off for example) any one of those machines and our object storage will still be available and ready to use for any purpose.

  • Jails

First we will create 3 jails for our proof of concept Minio setup, storage1 will have the ‘quorum/witness’ role while storage2 and storage3 will have the ‘data’ role. To distinguish commands I type on the host system and storageX Jail I use two different prompts, this way it should be obvious what command to execute and where.

  • WeI know the FreeNAS people have been working on integrating this

Best practises for pledge(2) security

Let's set the record straight for securing kcgi CGI and FastCGI applications with pledge(2). This is focussed on secure OpenBSD deployments.

  • Theory

Internally, kcgi makes considerable use of available security tools. But it's also designed to be invoked in a secure environment. We'll start with pledge(2), which has been around on OpenBSD since version 5.9. If you're reading this tutorial, you're probably on OpenBSD, and you probably have knowledge of pledge(2).

How to begin? Read kcgi(3). It includes canonical information on which pledge(2) promises you'll need for each function in the library. This is just a tutorial—the manpage is canonical and overrides what you may read here.

Next, assess the promises that your application needs. From kcgi(3), it's easy to see which promises we'll need to start. You'll need to augment this list with whichever tools you're also using. The general push is to start with the broadest set of required promises, then restrict as quickly as possible. Sometimes this can be done in a single pledge(2), but other times it takes a few.

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

May 17 2018

1hr 29mins

Play

Rank #15: 331: Why Computers Suck

Podcast cover
Read more

How learning OpenBSD makes computers suck a little less, How Unix works, FreeBSD 12.1 Runs Well on Ryzen Threadripper 3970X, BSDCan CFP, HardenedBSD Infrastructure Goals, and more.

Headlines

Why computers suck and how learning from OpenBSD can make them marginally less horrible

How much better could things actually be if we abandoned the enterprise development model?

Next I will compare this enterprise development approach with non-enterprise development - projects such as OpenBSD, which do not hesitate to introduce ABI breaking changes to improve the codebase.

One of the most commonly referred to pillars of the project's philosophy has long been its emphasis on clean functional code. Any code which makes it into OpenBSD is subject to ongoing aggressive audits for deprecated, or otherwise unmaintained code in order to reduce cruft and attack surface. Additionally the project creator, Theo de Raadt, and his team of core developers engage in ongoing development for proactive mitigations for various attack classes many of which are directly adopted by various multi-platform userland applications as well as the operating systems themselves (Windows, Linux, and the other BSDs). Frequently it is the case that introducing new features (not just deprecating old ones) introduces new incompatibilities against previously functional binaries compiled for OpenBSD.

To prevent the sort of kernel memory bloat that has plagued so many other operating systems for years, the project enforces a hard ceiling on the number of lines of code that can ever be in ring 0 at a given time. Current estimates guess the number of bugs per line of code in the Linux kernel are around 1 bug per every 10,000 lines of code. Think of this in the context of the scope creep seen in the Linux kernel (which if I recall correctly is currently at around 100,000,000 lines of code), as well as the Windows NT kernel (500,000,000 lines of code) and you quickly begin to understand how adding more and more functionality into the most privileged components of the operating system without first removing old components begins to add up in terms of the drastic difference seen between these systems in the number of zero day exploits caught in the wild respectively.

How Unix Works: Become a Better Software Engineer

Unix is beautiful. Allow me to paint some happy little trees for you. I’m not going to explain a bunch of commands – that’s boring, and there’s a million tutorials on the web doing that already. I’m going to leave you with the ability to reason about the system.

Every fancy thing you want done is one google search away.

But understanding why the solution does what you want is not the same.

That’s what gives you real power, the power to not be afraid.

And since it rhymes, it must be true.

News Roundup

FreeBSD 12.1 Runs Refreshingly Well With AMD Ryzen Threadripper 3970X

For those of you interested in AMD's new Ryzen Threadripper 3960X/3970X processors with TRX40 motherboards for running FreeBSD, the experience in our initial testing has been surprisingly pleasant. In fact, it works out-of-the-box which one could argue is better than the current Linux support that needs the MCE workaround for booting. Here are some benchmarks of FreeBSD 12.1 on the Threadripper 3970X compared to Linux and Windows for this new HEDT platform.

It was refreshing to see FreeBSD 12.1 booting and running just fine with the Ryzen Threadripper 3970X 32-core/64-thread processor from the ASUS ROG ZENITH II EXTREME motherboard and all core functionality working including the PCIe 4.0 NVMe SSD storage, onboard networking, etc. The system was running with 4 x 16GB DDR4-3600 memory, 1TB Corsair Force MP600 NVMe SSD, and Radeon RX 580 graphics. It was refreshing to see FreeBSD 12.1 running well with this high-end AMD Threadripper system considering Linux even needed a boot workaround.

While the FreeBSD 12.1 experience was trouble-free with the ASUS TRX40 motherboard (ROG Zenith II Extreme) and AMD Ryzen Threadripper 3970X, DragonFlyBSD unfortunately was not. Both DragonFlyBSD 5.6.2 stable and the DragonFlyBSD daily development snapshot from last week were yielding a panic on boot. So with that, DragonFlyBSD wasn't tested for this Threadripper 3970X comparison but just FreeBSD 12.1.

FreeBSD 12.1 on the Threadripper 3970X was benchmarked both with its default LLVM Clang 8.0.1 compiler and again with GCC 9.2 from ports for ruling out compiler differences. The FreeBSD 12.1 performance was compared to last week's Windows 10 vs. Linux benchmarks with the same system.

BSDCan 2020 CFP

BSDCan 2020 will be held 5-6 (Fri-Sat) June, 2020 in Ottawa, at the University of Ottawa. It will be preceded by two days of tutorials on 3-4 June (Wed-Thu).

NOTE the change of month in 2020 back to June Also: do not miss out on the Goat BOF on Tuesday 2 June.

We are now accepting proposals for talks. The talks should be designed with a very strong technical content bias. Proposals of a business development or marketing nature are not appropriate for this venue.

If you are doing something interesting with a BSD operating system, please submit a proposal. Whether you are developing a very complex system using BSD as the foundation, or helping others and have a story to tell about how BSD played a role, we want to hear about your experience. People using BSD as a platform for research are also encouraged to submit a proposal. Possible topics include:

  • How we manage a giant installation with respect to handling spam.
  • and/or sysadmin.
  • and/or networking.
  • Cool new stuff in BSD
  • Tell us about your project which runs on BSD
  • other topics (see next paragraph)

From the BSDCan website, the Archives section will allow you to review the wide variety of past BSDCan presentations as further examples.

Both users and developers are encouraged to share their experiences.

HardenedBSD Infrastructure Goals

2019 has been an extremely productive year with regards to HardenedBSD's infrastructure. Several opportunities aligned themselves in such a way as to open a door for a near-complete rebuild with a vast expansion.

The last few months especially have seen a major expansion of our infrastructure. We obtained a number of to-be-retired Dell R410 servers. The crash of our nightly build server provided the opportunity to deploy these R410 servers, doubling our build capacity.

My available time to spend on HardenedBSD has decreased compared to this time last year. As part of rebuilding our infrastructure, I wanted to enable the community to be able to contribute. I'm structuring the work such that help is just a pull request away. Those in the HardenedBSD community who want to contribute to the infrastructure work can simply open a pull request. I'll review the code, and deploy it after a successful review. Users/contributors don't need access to our servers in order to improve them.

My primary goal for the rest of 2019 and into 2020 is to become fully self-hosted, with the sole exception of email. I want to transition the source-of-truth git repos to our own infrastructure. We will still provide a read-only mirror on GitHub.

As I develop this infrastructure, I'm doing so with human rights in mind. HardenedBSD is in a very unique position. In 2020, I plan to provide production Tor Onion Services for the various bits of our infrastructure. HardenedBSD will provide access to its various internal services to its developers and contributors. The entire development lifecycle, going from dev to prod, will be able to happen over Tor.

Transparency will be key moving forward. Logs for the auto-sync script are now published directly to GitHub. Build logs will be, soon, too. Logs of all automated processes, and the code for those processes, will be tracked publicly via git. This will be especially crucial for development over Tor.

Integrating Tor into our infrastructure so deeply increases risk and maintenance burden. However, I believe that through added transparency, we will be able to mitigate risk. Periodic audits will need to be performed and published.

I hope to migrate HardenedBSD's site away from Drupal to a static site generator. We don't really need the dynamic capabilities Drupal gives us. The many security issues Drupal and PHP both bring also leave much to be desired.

So, that's about it. I spent the last few months of 2019 laying the foundation for a successful 2020. I'm excited to see how the project grows.

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

Jan 02 2020

1hr 9mins

Play

Rank #16: 295: Fun with funlinkat()

Podcast cover
Read more

Introducing funlinkat(), an OpenBSD Router with AT&T U-Verse, using NetBSD on a raspberry pi, ZFS encryption is still under development, Rump kernel servers and clients tutorial, Snort on OpenBSD 6.4, and more.

Headlines

Introducing funlinkat

  • It turns out, every file you have ever deleted on a unix machine was probably susceptible to a race condition

One of the first syscalls which was created in Unix-like systems is unlink. In FreeBSD this syscall is number 10 (source) and in Linux, the number is dependent on the architecture but for most of them is also the tenth syscall (source). This indicated that this is one of the primary syscalls. The unlink syscall is very simple and we provide one single path to the file that we want to remove.
The “removing file” process itself is very interesting so let’s spend a moment to understand the it. First, by removing the file we are removing a link from the directory to it. In Unix-like systems we can have many links to a single file (hard links). When we remove all links to the file, the file system will mark the blocks used by the file as free (a different file system will behave differently but let’s not jump into a second digression). This is why the process is called unlinking and not “removing file”. While we unlink the file two or three things will happen:

  • We will remove an entry in the directory with the filename.
  • We will decrease a file reference count (in inode).
  • If links go to zero - the file will be removed from the disk (again this doesn't mean that the blocks from the disk will be filled with zeros, though this may happen depending on the file system and configuration. However, in most cases this means that the file system will mark those blocks to as free and use them to write new data later
    This mostly means that “removing file” from a directory is an operation on the directory and not on the file (inode) itself.
    Another interesting subject is what happens if our system will perform only first or second step from the list. This depends on the file system and this is also something we will leave for another time.
    The problem with the unlink and even unlinkat function is that we don’t have any guarantee of which file we really are unlinking.
    • When you delete a file using its name, you have no guarantee that someone has not already deleted the file, or renamed it, and created a new file with the name you are about to delete.
      We have some stats about the file that we want to unlink. We performed some tests. In the same time another process removed our file and recreated it. When we finally try to remove our file it is no longer the same file. It’s a classic race condition.
    • Many programs will perform checks before trying to remove a file, to make sure it is the correct file, that you have the correct permissions etc. However this exposes the ‘Time-of-Check / Time-of-Use’ class of bugs. I check if the file I am about to remove is the one I created yesterday, it is, so I call unlink() on it. However, between when I checked the date on the file, and when I call unlink, I, some program I am running, might have updated the file. Or a malicious user might have put some other file at that name, so I would be the one who deleted it.
      In Unix-like operating systems we can get a handle for our file called file - a descriptor. File descriptors guarantee us that all the operations that we will be performing on it are done on the same file (inode). Even if someone was to unlink a number of directories entries, the operating system will not free the structures behind the file descriptor, and we can detect the file that was removed by someone and recreated (or just unlinked). So, for example, we have an alternative functions fstat which allows us to get file status of the given descriptor
      We already know that the file may have many links on the disk which point to the single inode. What happens when we open the file? Simplifying: kernel creates a memory representation of the inode (the inode itself is stored on the disk) called vnode. This single representation is used by all processes to refer the inode to the disk. If in a process we open the same file (inode) using different names (for example through hard links) all those files will be linked to the single vnode. That means that the pathname is not stored in the kernel.
      This is basically the reason why we don’t have a funlink function so that instead of the path we are providing just the file descriptor to the file. If we performed the fdunlink syscall, the kernel wouldn’t know which directory entry you would like to remove. Another problem is more architectural: as we discussed earlier unlinking is really an operation on the directory not on the file (inode) itself, so using funlink(fd) may create some confusion because we are not removing the inode corresponding to the file descriptor, we are performing action on the directory which points to the file.
      After some discussion we decided that the only sensible option for FreeBSD would be to create a funlinkat() function. This syscall would only performs additional sanitary checks if we are removing a directory entry which corresponds to the inode stored which refers to the file descriptor.
      int funlinkat(int dfd, const char *path, int fd, int flags);
      The API above will check if the path opened relative to the dfd points to the same vnode. Thanks to that we removed a race condition because all those sanitary checks are performed in the kernel mode while the file system is locked and there is no possibility to change it.
      The fd parameter may be set to the FD_NONE value which will mean that the sanitary check should not be performed and funlinkat will behave just like unlinkat.
      As you can notice I often refer to the unlink syscall but at the end the APIs looks like unlinkat syscall. It is true that the unlink syscall is very old and kind of deprecated. That said I referred to unlink because it’s just simpler. These days unlink simply uses the same code as unlinkat.

Using an OpenBSD Router with AT&T U-Verse

I upgraded to AT&T's U-verse Gigabit internet service in 2017 and it came with an Arris BGW-210 as the WiFi AP and router. The BGW-210 is not a terrible device, but I already had my own Airport Extreme APs wired throughout my house and an OpenBSD router configured with various things, so I had no use for this device. It's also a potentially-insecure device that I can't upgrade or fully disable remote control over.
Fully removing the BGW-210 is not possible as we'll see later, but it is possible to remove it from the routing path. This is how I did it with OpenBSD.

News Roundup

How to use NetBSD on a Raspberry Pi

Do you have an old Raspberry Pi lying around gathering dust, maybe after a recent Pi upgrade? Are you curious about BSD Unix? If you answered "yes" to both of these questions, you'll be pleased to know that the first is the solution to the second, because you can run NetBSD, as far back as the very first release, on a Raspberry Pi.
BSD is the Berkley Software Distribution of Unix. In fact, it's the only open source Unix with direct lineage back to the original source code written by Dennis Ritchie and Ken Thompson at Bell Labs. Other modern versions are either proprietary (such as AIX and Solaris) or clever re-implementations (such as Minix and GNU/Linux). If you're used to Linux, you'll feel mostly right at home with BSD, but there are plenty of new commands and conventions to discover. If you're still relatively new to open source, trying BSD is a good way to experience a traditional Unix.
Admittedly, NetBSD isn't an operating system that's perfectly suited for the Pi. It's a minimal install compared to many Linux distributions designed specifically for the Pi, and not all components of recent Pi models are functional under NetBSD yet. However, it's arguably an ideal OS for the older Pi models, since it's lightweight and lovingly maintained. And if nothing else, it's a lot of fun for any die-hard Unix geek to experience another side of the POSIX world.

ZFS Encryption is still under development (as of March 2019)

One of the big upcoming features that a bunch of people are looking forward to in ZFS is natively encrypted filesystems. This is already in the main development tree of ZFS On Linux, will likely propagate to FreeBSD (since FreeBSD ZFS will be based on ZoL), and will make it to Illumos if the Illumos people want to pull it in. People are looking forward to native encryption so much, in fact, that some of them have started using it in ZFS On Linux already, using either the development tip or one of the 0.8.0 release candidate pre-releases (ZoL is up to 0.8.0-rc3 as of now). People either doing this or planning to do this show up on the ZoL mailing list every so often.

Tutorial On Rump Kernel Servers and Clients

The rump anykernel architecture allows to run highly componentized kernel code configurations in userspace processes. Coupled with the rump sysproxy facility it is possible to run loosely distributed client-server "mini-operating systems". Since there is minimum configuration and the bootstrap time is measured in milliseconds, these environments are very cheap to set up, use, and tear down on-demand.
This document acts as a tutorial on how to configure and use unmodified NetBSD kernel drivers as userspace services with utilities available from the NetBSD base system. As part of this, it presents various use cases. One uses the kernel cryptographic disk driver (cgd) to encrypt a partition. Another one demonstrates how to operate an FFS server for editing the contents of a file system even though your user account does not have privileges to use the host's mount() system call. Additionally, using a userspace TCP/IP server with an unmodified web browser is detailed.

Installing Snort on OpenBSD 6.4

As you may recall from previous posts, I am running an OpenBSD server on an APU2 air-cooled 3 Intel NIC box as my router/firewall for my secure home network. Given that all of my Internet traffic flows through this box, I thought it would be a cool idea to run an Intrusion Detection System (IDS) on it. Snort is the big hog of the open source world so I took a peek in the packages directory on one of the mirrors and lo and behold we have the latest & greatest version of Snort available! Thanks devs!!!
I did some quick Googling and didn’t find much “modern” howto help out there so, after some trial and error, I have it up and running. I thought I’d give back in a small way and share a quickie howto for other Googlers out there who are looking for guidance. Here’s hoping that my title is good enough “SEO” to get you here!

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv


Your browser does not support the HTML5 video tag.

Apr 25 2019

1hr 1min

Play

Rank #17: 330: Happy Holidays, All(an)

Podcast cover
Read more

Authentication Vulnerabilities in OpenBSD, NetBSD 9.0 RC1 is available, Running FreeNAS on a DigitalOcean droplet, NomadBSD 1.3 is here, at e2k19 nobody can hear you scream, and more.

Headlines

Authentication vulnerabilities in OpenBSD

  • We discovered an authentication-bypass vulnerability in OpenBSD's authentication system: this vulnerability is remotely exploitable in smtpd, ldapd, and radiusd, but its real-world impact should be studied on a case-by-case basis. For example, sshd is not exploitable thanks to its defense-in-depth mechanisms.
  • From the manual page of login.conf:

OpenBSD uses BSD Authentication, which is made up of a variety of authentication styles. The authentication styles currently provided are:
passwd Request a password and check it against the password in the master.passwd file. See login_passwd(8).
skey Send a challenge and request a response, checking it with S/Key (tm) authentication. See login_skey(8).
yubikey Authenticate using a Yubico YubiKey token. See login_yubikey(8).
For any given style, the program /usr/libexec/auth/login_style is used to
perform the authentication. The synopsis of this program is:
/usr/libexec/auth/login_style [-v name=value] [-s service] username class

  • This is the first piece of the puzzle: if an attacker specifies a username of the form "-option", they can influence the behavior of the authentication program in unexpected ways.
login_passwd [-s service] [-v wheel=yes|no] [-v lastchance=yes|no] user [class] The service argument specifies which protocol to use with the invoking program. The allowed protocols are login, challenge, and response. (The challenge protocol is silently ignored but will report success as passwd-style authentication is not challenge-response based).
  • This is the second piece of the puzzle: if an attacker specifies the username "-schallenge" (or "-schallenge:passwd" to force a passwd-style authentication), then the authentication is automatically successful and therefore bypassed.
  • Case study: smtpd
  • Case study: ldapd
  • Case study: radiusd
  • Case study: sshd
  • Acknowledgments: We thank Theo de Raadt and the OpenBSD developers for their incredibly quick response: they published patches for these vulnerabilities less than 40 hours after our initial contact. We also thank MITRE's CVE Assignment Team.

First release candidate for NetBSD 9.0 available!

  • Since the start of the release process four months ago a lot of improvements went into the branch - more than 500 pullups were processed!
  • This includes usbnet (a common framework for usb ethernet drivers), aarch64 stability enhancements and lots of new hardware support, installer/sysinst fixes and changes to the NVMM (hardware virtualization) interface.
  • We hope this will lead to the best NetBSD release ever (only to be topped by NetBSD 10 next year).
  • Here are a few highlights of the new release:

    Support for Arm AArch64 (64-bit Armv8-A) machines, including "Arm ServerReady"
    compliant machines (SBBR+SBSA)
    Enhanced hardware support for Armv7-A
    Updated GPU drivers (e.g. support for Intel Kabylake)
    Enhanced virtualization support
    Support for hardware-accelerated virtualization (NVMM)
    Support for Performance Monitoring Counters
    Support for Kernel ASLR
    Support several kernel sanitizers (KLEAK, KASAN, KUBSAN)
    Support for userland sanitizers
    Audit of the network stack
    Many improvements in NPF
    Updated ZFS
    Reworked error handling and NCQ support in the SATA subsystem
    Support a common framework for USB Ethernet drivers (usbnet)

  • More information on the RC can be found on the NetBSD 9 release page

News Roundup

Running FreeNAS on a Digitalocean droplet

  • ZFS is awesome. FreeBSD even more so. FreeNAS is the battle-tested, enterprise-ready-yet-home-user-friendly software defined storage solution which is cooler then deep space, based on FreeBSD and makes heavy use of ZFS. This is what I (and soooooo many others) use for just about any storage-related task. I can go on and on and on about what makes it great, but if you're here, reading this, you probably know all that already and we can skip ahead.
  • I've needed an offsite FreeNAS setup to replicate things to, to run some things, to do some stuff, basically, my privately-owned, tightly-controlled NAS appliance in the cloud, one I control from top to bottom and with support for whatever crazy thing I'm trying to do. Since I'm using DigitalOcean as my main VPS provider, it seemed logical to run FreeNAS there, however, you can't. While DO supports many many distos and pre-setup applications (e.g OpenVPN), FreeNAS isn't a supported feature, at least not in the traditional way :)
  • Before we begin, here's the gist of what we're going to do:

Base of a FreeBSD droplet, we'll re-image our boot block device with FreeNAS iso. We'll then install FreeNAS on the second block device. Once done we're going to do the ol' switcheroo: we're going to re-image our original boot block device using the now FreeNAS-installed second block device.

  • Part 1: re-image our boot block device to boot FreeNAS install media.
  • Part 2: Install FreeNAS on the second block-device
  • Part 3: Re-image the boot block device using the FreeNAS-installed block device

NomadBSD 1.3 is now available

  • From the release notes:

The base system has been changed to FreeBSD 12.1-RELEASE-p1
Due to a deadlock problem, FreeBSD's unionfs has been replaced by unionfs-fuse
The GPT layout has been changed to MBR. This prevents problems with Lenovo
systems that refuse to boot from GPT if "lenovofix" is not set, and systems that
hang on boot if "lenovofix" is set.
Support for ZFS installations has been added to the NomadBSD installer.
The rc-script for setting up the network interfaces has been fixed and improved.
Support for setting the country code for the wlan device has been added.
Auto configuration for running in VirtualBox has been added.
A check for the default display has been added to the graphics configuration scripts. This fixes problems where users with Optimus have their NVIDIA card disabled, and use the integrated graphics chip instead.
NVIDIA driver version 440 has been added.
nomadbsd-dmconfig, a Qt tool for selecting the display manager theme, setting the
default user and autologin has been added.
nomadbsd-adduser, a Qt tool for added preconfigured user accounts to the system has been added.
Martin Orszulik added Czech translations to the setup and installation wizard.
The NomadBSD logo, designed by Ian Grindley, has been changed.
Support for localized error messages has been added.
Support for localizing the password prompts has been added.
Some templates for starting other DEs have been added to ~/.xinitrc.
The interfaces of nomadbsd-setup-gui and nomadbsd-install-gui have been improved.
A script that helps users to configure a multihead systems has been added.
The Xorg driver for newer Intel GPUs has been changed from "intel" to "modesetting".
/proc has been added to /etc/fstab
A D-Bus session issue has been fixed which prevented thunar from accessing samba shares.
DSBBg which allows users to change and manage wallpapers has been added.
The latest version of update_obmenu now supports auto-updating the Openbox menu. Manually updating the Openbox menu after packet (de)installation is therefore no longer needed.

Support for multiple keyboard layouts has been added.
www/palemoon has been removed.
mail/thunderbird has been removed.
audio/audacity has been added.
deskutils/orage has been added.
the password manager fpm2 has been replaced by KeePassXC
mail/sylpheed has been replaced by mail/claws-mail
multimedia/simplescreenrecorder has been added.
DSBMC has been changed to DSBMC-Qt
Many small improvements and bug fixes.

At e2k19 nobody can hear you scream

  • After 2 years it was once again time to pack skis and snowshoes, put a satellite dish onto a sledge and hike through the snowy rockies to the Elk Lakes hut.
  • I did not really have much of a plan what I wanted to work on but there were a few things I wanted to look into. One of them was rpki-client and the fact that it was so incredibly slow. Since Bob beck@ was around I started to ask him innocent X509 questions ... as if there are innocent X509 questions! Mainly about the abuse of the X509_STORE in rpki-client. Pretty soon it was clear that rpki-client did it all wrong and most of the X509 verification had to be rewritten. Instead of only storing the root certificates in the store and passing the intermediate certs as a chain to the verification function rpki-client threw everything into it. The X509_STORE is just not built for such an abuse and so it was no wonder that this was slow.
  • Lucky me I pulled benno@ with me into this dark hole of libcrypto code. He managed to build up an initial diff to pass the chains as a STACK_OF(X509) and together we managed to get it working. A big thanks goes to ingo@ who documented most of the functions we had to use. Have a look at STACK_OF(3) and sk_pop_free(3) to understand why benno@ and I slowly turned crazy.
  • Our next challenge was to only load the necessary certificate revocation list into the X509_STORE_CTX. While doing those changes it became obvious that some of the data structures needed better lookup functions. Looking up certificates was done using a linear lookup and so we replaced the internal certificate and CRL tables with RB trees for fast lookups. deraadt@ also joined the rpki-client commit fest and changed the output code to use rename(2) so that files are replaced in an atomic operation. Thanks to this rpki-client can now be safely run from cron (there is an example in the default crontab).
  • I did not plan to spend most of my week hacking on rpki-client but in the end I'm happy that I did and the result is fairly impressive. Working with libcrypto code and especially X509 was less than pleasant. Our screams of agony died away in the snowy rocky mountains and made Bob deep dive into UVM with a smile since he knew that benno@ and I had it worse.
  • In case you wonder thanks to all changes at e2k19 rpki-client improved from over 20min run time to validate all VRPS to roughly 1min to do the same job. A factor 20 improvement!
  • Thanks to Theo, Bob and Howie to make this possible. To all the cooks for the great food and to Xplornet for providing us with Internet at the hut.

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Your browser does not support the HTML5 video tag.

Special Guest: Mariusz Zaborski.

Dec 26 2019

1hr 15mins

Play

Rank #18: 302: Contention Reduction

Podcast cover
Read more

DragonFlyBSD's kernel optimizations pay off, differences between OpenBSD and Linux, NetBSD 2019 Google Summer of Code project list, Reducing that contention, fnaify 1.3 released, vmctl(8): CLI syntax changes, and things that Linux distributions should not do when packaging.

Headlines

DragonFlyBSD's Kernel Optimizations Are Paying Off

DragonFlyBSD lead developer Matthew Dillon has been working on a big VM rework in the name of performance and other kernel improvements recently. Here is a look at how those DragonFlyBSD 5.5-DEVELOPMENT improvements are paying off compared to DragonFlyBSD 5.4 as well as FreeBSD 12 and five Linux distribution releases. With Dillon using an AMD Ryzen Threadripper system, we used that too for this round of BSD vs. Linux performance benchmarks.
The work by Dillon on the VM overhaul and other changes (including more HAMMER2 file-system work) will ultimately culminate with the DragonFlyBSD 5.6 release (well, unless he opts for DragonFlyBSD 6.0 or so). These are benchmarks of the latest DragonFlyBSD 5.5-DEVELOPMENT daily ISO as of this week benchmarked across DragonFlyBSD 5.4.3 stable, FreeBSD 12.0, Ubuntu 19.04, Red Hat Enterprise Linux 8.0, Debian 9.9, Debian Buster, and CentOS 7 1810 as a wide variety of reference points both from newer and older Linux distributions. (As for no Clear Linux reference point for a speedy reference point, it currently has a regression with AMD + Samsung NVMe SSD support on some hardware, including this box, prohibiting the drive from coming up due to a presumed power management issue that is still being resolved.)
With Matthew Dillon doing much of his development on an AMD Ryzen Threadripper system after he last year proclaimed the greatness of these AMD HEDT CPUs, for this round of testing I also used a Ryzen Threadripper 2990WX with 32 cores / 64 threads. Tests of other AMD/Intel hardware with DragonFlyBSD will come as the next stable release is near and all of the kernel work has settled down. For now it's mostly entertaining our own curiosity how well these DragonFlyBSD optimizations are paying off and how it's increasing the competition against FreeBSD 12 and Linux distributions.

What are the differences between OpenBSD and Linux?

Maybe you have been reading recently about the release of OpenBSD 6.5 and wonder, "What are the differences between Linux and OpenBSD?"
I've also been there at some point in the past and these are my conclusions.
They also apply, to some extent, to other BSDs. However, an important disclaimer applies to this article.
This list is aimed at people who are used to Linux and are curious about OpenBSD. It is written to highlight the most important changes from their perspective, not the absolute most important changes from a technical standpoint.
Please bear with me.

  • A terminal is a terminal is a terminal
  • Practical differences
  • Security and system administration
  • Why philosophical differences matter
  • So what do I choose?
  • How to try OpenBSD ***

News Roundup

NetBSD 2019 Google Summer of Code

We are very happy to announce The NetBSD Foundation Google Summer of Code 2019 projects:

  • Akul Abhilash Pillai - Adapting TriforceAFL for NetBSD kernel fuzzing
  • Manikishan Ghantasala - Add KNF (NetBSD style) clang-format configuration
  • Siddharth Muralee - Enhancing Syzkaller support for NetBSD
  • Surya P - Implementation of COMPAT_LINUX and COMPAT_NETBSD32 DRM ioctls support for NetBSD kernel
  • Jason High - Incorporation of Argon2 Password Hashing Algorithm into NetBSD
  • Saurav Prakash - Porting NetBSD to HummingBoard Pulse
  • Naveen Narayanan - Porting WINE to amd64 architecture on NetBSD

The communiting bonding period - where students get in touch with mentors and community - started yesterday. The coding period will start from May 27 until August 19.
Please welcome all our students and a big good luck to students and mentors! A big thank to Google and The NetBSD Foundation organization mentors and administrators! Looking forward to a great Google Summer of Code!

Reducing that contention

The opening keynote at EuroBSDCon 2016 predicted the future 10 years of BSDs. Amongst all the funny previsions, gnn@FreeBSD said that by 2026 OpenBSD will have its first implementation of SMP. Almost 3 years after this talk, that sounds like a plausible forecast... Why? Where are we? What can we do? Let's dive into the issue!

  • State of affairs

Most of OpenBSD's kernel still runs under a single lock, ze KERNEL_LOCK(). That includes most of the syscalls, most of the interrupt handlers and most of the fault handlers. Most of them, not all of them. Meaning we have collected & fixed bugs while setting up infrastructures and examples. Now this lock remains the principal responsible for the spin % you can observe in top(1) and systat(1).
I believe that we opted for a difficult hike when we decided to start removing this lock from the bottom. As a result many SCSI & Network interrupt handlers as well as all Audio & USB ones can be executed without big lock. On the other hand very few syscalls are already or almost ready to be unlocked, as we incorrectly say. This explains why basic primitives like tsleep(9), csignal() and selwakeup() are only receiving attention now that the top of the Network Stack is running (mostly) without big lock.

  • Next steps

In the past years, most of our efforts have been invested into the Network Stack. As I already mentioned it should be ready to be parallelized. However think we should now concentrate on removing the KERNEL_LOCK(), even if the code paths aren't performance critical.

  • See the Article for the rest of the post

fnaify 1.3 released - more games are "fnaify & run" now

This release finally addresses some of the problems that prevent simple running of several games.
This happens for example when an old FNA.dll library comes with the games that doesn't match the API of our native libraries like SDL2, OpenAL, or MojoShader anymore. Some of those cases can be fixed by simply dropping in a newer FNA.dll. fnaify now asks if FNA 17.12 should be automatically added if a known incompatible FNA version is found. You simply answer yes or no.

Another blocker happens when the game expects to check the SteamAPI - either from a running Steam process, or a bundled steam_api library. OpenBSD 6.5-current now has steamworks-nosteam in ports, a stub library for Steamworks.NET that prevents games from crashing simply because an API function isn't found. The repo is here. fnaify now finds this library in /usr/local/share/steamstubs and uses it instead of the bundled (full) Steamworks.NET.dll.
This may help with any games that use this layer to interact with the SteamAPI, mostly those that can only be obtained via Steam.

vmctl(8): command line syntax changed

The order of the arguments in the create, start, and stop commands of vmctl(8) has been changed to match a commonly expected style. Manual usage or scripting with vmctl must be adjusted to use the new syntax.
For example, the old syntax looked like this:

# vmctl create disk.qcow2 -s 50G

The new syntax specifies the command options before the argument:

# vmctl create -s 50G disk.qcow2

Something that Linux distributions should not do when packaging things

Right now I am a bit unhappy at Fedora for a specific packaging situation, so let me tell you a little story of what I, as a system administrator, would really like distributions to not do.
For reasons beyond the scope of this blog entry, I run a Prometheus and Grafana setup on both my home and office Fedora Linux machines (among other things, it gives me a place to test out various things involving them). When I set this up, I used the official upstream versions of both, because I needed to match what we are running (or would soon be).
Recently, Fedora decided to package Grafana themselves (as a RPM), and they called this RPM package 'grafana'. Since the two different packages are different versions of the same thing as far as package management tools are concerned, Fedora basically took over the 'grafana' package name from Grafana. This caused my systems to offer to upgrade me from the Grafana.com 'grafana-6.1.5-1' package to the Fedora 'grafana-6.1.6-1.fc29' one, which I actually did after taking reasonable steps to make sure that the Fedora version of 6.1.6 was compatible with the file layouts and so on from the Grafana version of 6.1.5.
Why is this a problem? It's simple. If you're going to take over a package name from the upstream, you should keep up with the upstream releases. If you take over a package name and don't keep up to date or keep up to date only sporadically, you cause all sorts of heartburn for system administrators who use the package. The least annoying future of this situation is that Fedora has abandoned Grafana at 6.1.6 and I am going to 'upgrade' it with the upstream 6.2.1, which will hopefully be a transparent replacement and not blow up in my face. The most annoying future is that Fedora and Grafana keep ping-ponging versions back and forth, which will make 'dnf upgrade' into a minefield (because it will frequently try to give me a 'grafana' upgrade that I don't want and that would be dangerous to accept). And of course this situation turns Fedora version upgrades into their own minefield, since now I risk an upgrade to Fedora 30 actually reverting the 'grafana' package version on me.

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv ***
Your browser does not support the HTML5 video tag.

Jun 13 2019

1hr 9mins

Play

Rank #19: 297: Dragonfly In The Wild

Podcast cover
Read more

FreeBSD ZFS vs. ZoL performance, Dragonfly 5.4.2 has been release, containing web services with iocell, Solaris 11.4 SRU8, Problem with SSH Agent forwarding, OpenBSD 6.4 to 6.5 upgrade guide, and more.

Headlines

FreeBSD ZFS vs. ZoL Performance, Ubuntu ZFS On Linux Reference

With iX Systems having released new images of FreeBSD reworked with their ZFS On Linux code that is in development to ultimately replace their existing FreeBSD ZFS support derived from the code originally found in the Illumos source tree, here are some fresh benchmarks looking at the FreeBSD 12 performance of ZFS vs. ZoL vs. UFS and compared to Ubuntu Linux on the same system with EXT4 and ZFS.
Using an Intel Xeon E3-1275 v6 with ASUS P10S-M WS motherboard, 2 x 8GB DDR4-2400 ECC UDIMMs, and Samsung 970 EVO Plus 500GB NVMe solid-state drive was used for all of this round of testing. Just a single modern NVMe SSD was used for this round of ZFS testing while as the FreeBSD ZoL code matures I'll test on multiple systems using a more diverse range of storage devices.
FreeBSD 12 ZoL was tested using the iX Systems image and then fresh installs done of FreeBSD 12.0-RELEASE when defaulting to the existing ZFS root file-system support and again when using the aging UFS file-system. Ubuntu 18.04.2 LTS with the Linux 4.18 kernel was used when testing its default EXT4 file-system and then again when using the Ubuntu-ZFS ZoL support. Via the Phoronix Test Suite various BSD/Linux I/O benchmarks were carried out.
Overall, the FreeBSD ZFS On Linux port is looking good so far and we are looking forward to it hopefully maturing in time for FreeBSD 13.0. Nice job to iX Systems and all of those involved, especially the ZFS On Linux project. Those wanting to help in testing can try the FreeBSD ZoL spins. Stay tuned for more benchmarks and on more diverse hardware as time allows and the FreeBSD ZoL support further matures, but so far at least the performance numbers are in good shape.

DragonFlyBSD 5.4.2 is out

Upgrading guide

Here's the tag commit, for what has changed from 5.4.1 to 5.4.2
The normal ISO and IMG files are available for download and install, plus an uncompressed ISO image for those installing remotely. I uploaded them to mirror-master.dragonflybsd.org last night so they should be at your local mirror or will be soon. This version includes Matt's fix for the HAMMER2 corruption bug he identified recently.
If you have an existing 5.4 system and are running a generic kernel, the normal upgrade process will work.

> cd /usr/src
> git pull
> make buildworld.
> make buildkernel.
> make installkernel.
> make installworld
> make upgrade

After your next reboot, you can optionally update your rescue system:

> cd /usr/src
> make initrd

As always, make sure your packages are up to date:

> pkg update
> pkg upgrade

News Roundup

Containing web services with iocell

I'm a huge fan of the FreeBSD jails feature. It is a great system for splitting services into logical units with all the performance of the bare metal system. In fact, this very site runs in its own jail! If this is starting to sound like LXC or Docker, it might surprise you to learn that OS-level virtualization has existed for quite some time. Kudos to the Linux folks for finally getting around to it. 😛
If you're interested in the history behind Jails, there is an excellent talk from Papers We Love on the subject: https://www.youtube.com/watch?v=hgN8pCMLI2U

  • Getting started

There are plenty of options when it comes to setting up the jail system. Ezjail and Iocage seem popular, or you could do things manually. Iocage was recently rewritten in python, but was originally a set of shell scripts. That version has since been forked under the name Iocell, and I think it's pretty neat, so this tutorial will be using Iocell.

  • To start, you'll need the following:
    • A FreeBSD install (we'll be using 11.0)
    • The iocell package (available as a package, also in the ports tree)
    • A ZFS pool for hosting the jails

Once you have installed iocell and configured your ZFS pool, you'll need to run a few commands before creating your first jail. First, tell iocell which ZFS pool to use by issuing iocell activate $POOLNAME. Iocell will create a few datasets.

As you can imagine, your jails are contained within the /iocell/jails dataset. The /iocell/releases dataset is used for storing the next command we need to run, iocell fetch. Iocell will ask you which release you'd like to pull down. Since we're running 11.0 on the host, pick 11.0-RELEASE. Iocell will download the necessary txz files and unpack them in /iocell/releases.

  • See Article for the rest of the walkthrough.

Oracle Solaris 11.4 SRU8

Today we are releasing the SRU 8 for Oracle Solaris 11.4. It is available via 'pkg update' from the support repository or by downloading the SRU from My Oracle Support Doc ID 2433412.1.

  • This SRU introduces the following enhancements:
    • Integration of 28060039 introduced an issue where any firmware update/query commands will log eereports and repeated execution of such commands led to faulty/degraded NIC. The issue has been addressed in this SRU.
    • UCB (libucb, librpcsoc, libdbm, libtermcap, and libcurses) libraries have been reinstated for Oracle Solaris 11.4
    • Re-introduction of the service fc-fabric.
    • ibus has been updated to 1.5.19
  • The following components have also been updated to address security issues:
    • NTP has been updated to 4.2.8p12
    • Firefox has been updated to 60.6.0esr
    • BIND has been updated to 9.11.6
    • OpenSSL has been updated to 1.0.2r
    • MySQL has been updated to 5.6.43 & 5.7.25
    • libxml2 has been updated to 2.9.9
    • libxslt has been updated to 1.1.33
    • Wireshark has been updated to 2.6.7
    • ncurses has been updated to 6.1.0.20190105
    • Apache Web Server has been updated to 2.4.38
    • perl 5.22
    • pkg.depot

The Problem with SSH Agent Forwarding

After hacking the matrix.org website today, the attacker opened a series of GitHub issues mentioning the flaws he discovered. In one of those issues, he mentions that “complete compromise could have been avoided if developers were prohibited from using [SSH agent forwarding].”
Here’s what man ssh_config has to say about ForwardAgent: "Agent forwarding should be enabled with caution. Users with the ability to bypass file permissions on the remote host (for the agent’s Unix-domain socket) can access the local agent through the forwarded connection. An attacker cannot obtain key material from the agent, however they can perform operations on the keys that enable them to authenticate using the identities loaded into the agent.""
Simply put: if your jump box is compromised and you use SSH agent forwarding to connect to another machine through it, then you risk also compromising the target machine!
Instead, you should use either ProxyCommand or ProxyJump (added in OpenSSH 7.3). That way, ssh will forward the TCP connection to the target host via the jump box and the actual connection will be made on your workstation. If someone on the jump box tries to MITM your connection, then you will be warned by ssh.

[OpenBSD Upgrade Guide: 6.4 to 6.5

Start by performing the pre-upgrade steps. Next, boot from the install kernel, bsd.rd: use bootable install media, or place the 6.5 version of bsd.rd in the root of your filesystem and instruct the boot loader to boot this kernel. Once this kernel is booted, choose the (U)pgrade option and follow the prompts. Apply the configuration changes and remove the old files. Finish up by upgrading the packages: pkg_add -u.
Alternatively, you can use the manual upgrade process.
You may wish to check the errata page or upgrade to the stable branch to get any post-release fixes.

  • Before rebooting into the install kernel
  • Configuration and syntax changes
  • Files to remove
  • Special packages
  • Upgrade without the install kernel

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv


Your browser does not support the HTML5 video tag.

May 09 2019

40mins

Play

Rank #20: 293: Booking Jails

Podcast cover
Read more

This week we have a special episode with a Michael W. Lucas interview about his latest jail book that’s been released. We’re talking all things jails, writing, book sponsoring, the upcoming BSDCan 2019 conference, and more.

###Interview - Michael W. Lucas - mwl@mwl.io / @mwlauthor
FreeBSD Mastery: Jails

  • BR: Welcome back to the show and congratulations on your latest book. How many books did you have to write before you could start on FreeBSD Mastery: Jails?
  • AJ: How much research did you have to do about jails?
  • BR: The book talks about something called ‘incomplete’ jails. What do you mean by that?
  • AJ: There are a lot of jail management frameworks out there. Why did you chose to write about iocage in the book?
  • BR: How many jails do you run yourself?
  • AJ: Can you tell us a bit about how you handle book sponsorship these days?
  • BR: What other books (fiction and non-fiction) are you currently working on?
  • AJ: Which talks are you looking forward to attend at the upcoming BSDCan conference?
  • BR: How is the BSD user group going?
  • AJ: Anything else you’d like to mention before we release you from our interview jail cell?

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

Apr 11 2019

1hr 16mins

Play

362: 2.11-BSD restoration

Podcast cover
Read more

Interview with Warner Losh about Unix history, the 2.11-BSD restoration project, the Unix heritage society, proper booting, and what devmatch is.

Interview - Warner Losh - imp@freebsd.org / @bsdimp

BSD 2.11 restoration project

Tarsnap

  • This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Special Guest: Warner Losh.

Aug 06 2020

1hr 2mins

Play

361: Function-based MicroVM

Podcast cover
Read more

Emulex: The Cheapest 10gbe for Your Homelab, In Search of 2.11BSD, as released, Fakecracker: NetBSD as a Function Based MicroVM, First powerpc64 snapshots available for OpenBSD, OPNsense 20.1.8 released, and more.

NOTES
This episode of BSDNow is brought to you by Tarsnap

Headlines

Emulex: The Cheapest 10gbe for Your Homelab

Years ago, the hunt for the cheapest 10gbe NICs resulted in buying Mellanox ConnectX-2 single-port 10gbe network cards from eBay for around $10. Nowadays those cards have increased in cost to around $20-30. While still cheap, not quite the cheapest. There are now alternatives!
Before diving into details, let’s get something very clear. If you want the absolute simplest plug-and-play 10gbe LAN for your homelab, pay the extra for Mellanox. If you’re willing to go hands-on, do some simple manual configuration and installation, read on for my experiences with Emulex 10gbe NICs.
Emulex NICs can often be had for around $15 on eBay, sometimes even cheaper. I recently picked up a set of 4 of these cards, which came bundled with 6 SFP+ 10g-SR modules for a grand total of $47.48. Considering I can usually find SFP+ modules for about $5/ea, these alone were worth $30.

  • I have also tried some Solarflare cards that I found cheap, they work ok, but are pickier about optics, and tend to be focused on low-latency, so often don’t manage to saturate the full 10 gbps, topping out around 8 gbps.
  • I have been using fs.com for optics, patch cables, and DACs. I find DACs are usually cheaper if you are just going between a server and a switch in the same rack, or direct between 2 servers. ***

In Search of 2.11BSD, as released

Almost all of the BSD releases have been well preserved. If you want to find 1BSD, or 2BSD or 4.3-TAHOE BSD you can find them online with little fuss. However, if you search for 2.11BSD, you'll find it easily enough, but it won't be the original. You'll find either the latest patched version (2.11BSD pl 469), or one of the earlier popular version (pl 430 is popular). You can even find the RetroBSD project which used 2.11BSD as a starting point to create systems for tiny mips-based PIC controllers. You'll find every single patch that's been issued for the system.

News Roundup

Fakecracker: NetBSD as a Function Based MicroVM

In November 2018 AWS published an Open Source tool called Firecracker, mostly a virtual machine monitor relying on KVM, a small sized Linux kernel, and a stripped down version of Qemu. What baffled me was the speed at which the virtual machine would fire up and run the service. The whole process is to be compared to a container, but safer, as it does not share the kernel nor any resource, it is a separate and dedicated virtual machine.
If you want to learn more on Firecracker‘s internals, here’s a very well put article.

First powerpc64 snapshots available for OpenBSD

Since we reported the first bits of powerpc64 support going into the tree on 16 May, work has progressed at a steady pace, resulting in snapshots now being available for this platform.
So, if you have a POWER9 system idling around, go to your nearest mirror and fetch this snapshot. Keep in mind that as this is still very early days, very little handholding is available - you are basically on your own.

OPNsense 20.1.8 released

Sorry about the delay while we chased a race condition in the updates back to an issue with the latest FreeBSD package manager updates. For now we reverted to our current version but all relevant third party packages have been updated as updates became available over the last weeks, e.g. cURL and Python, and hostapd / wpa_supplicant amongst others.

Beastie Bits

Tarsnap

  • This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv ***

Special Guest: Warner Losh.

Jul 30 2020

1hr 2mins

Play

360: Full circle

Podcast cover
Read more

Chasing a bad commit, New FreeBSD Core Team elected, Getting Started with NetBSD on the Pinebook Pro, FreeBSD on the Intel 10th Gen i3 NUC, pf table size check and change, and more.

NOTES
This episode of BSDNow is brought to you by Tarsnap

Headlines

Chasing a bad commit

While working on a big project where multiple teams merge their feature branches frequently into a release Git branch, developers often run into situations where they find that some of their work have been either removed, modified or affected by someone else's work accidentally. It can happen in smaller teams as well. Two features could have been working perfectly fine until they got merged together and broke something. That's a highly possible case. There are many other cases which could cause such hard to understand and subtle bugs which even continuous integration (CI) systems running the entire test suite of our projects couldn't catch.
We are not going to discuss how such subtle bugs can get into our release branch because that's just a wild territory out there. Instead, we can definitely discuss about how to find a commit that deviated from an expected outcome of a certain feature. The deviation could be any behaviour of our code that we can measure distinctively — either good or bad in general.

New FreeBSD Core Team Elected

The FreeBSD Project is pleased to announce the completion of the 2020 Core Team election. Active committers to the project have elected your Eleventh FreeBSD Core Team.!

  • Baptiste Daroussin (bapt)
  • Ed Maste (emaste)
  • George V. Neville-Neil (gnn)
  • Hiroki Sato (hrs)
  • Kyle Evans (kevans)
  • Mark Johnston (markj)
  • Scott Long (scottl)
  • Sean Chittenden (seanc)
  • Warner Losh (imp) ***

News Roundup

Getting Started with NetBSD on the Pinebook Pro

If you buy a Pinebook Pro now, it comes with Manjaro Linux on the internal eMMC storage. Let’s install NetBSD instead!
The easiest way to get started is to buy a decent micro-SD card (what sort of markings it should have is a science of its own, by the way) and install NetBSD on that. On a warm boot (i.e. when rebooting a running system), the micro-SD card has priority compared to the eMMC, so the system will boot from there.

FreeBSD on the Intel 10th Gen i3 NUC

I have ended up with some 10th Gen i3 NUC's (NUC10i3FNH to be specific) to put to work in my testbed. These are quite new devices, the build date on the boxes is 13APR2020. Before I figure out what their true role is (one of them might have to run linux) I need to install FreeBSD -CURRENT and see how performance and hardware support is.

pf table size check and change

Did you know there’s a default size limit to pf’s state table? I did not, but it makes sense that there is one. If for some reason you bump into this limit (difficult for home use, I’d think), here’s how you change it
There is a table-entries limit specified, you can see current settings with
'pfctl -s all'. You can adjust the limits in the /etc/pf.conf file
containing the rules with a line like this near the top:
set limit table-entries 100000

  • In the original mail thread, there is mention of the FreeBSD sysctl net.pf.request_maxcount, which controls the maximum number of entries that can be sent as a single ioctl(). This allows the user to adjust the memory limit for how big of a list the kernel is willing to allocate memory for. ***

Beastie Bits

Feedback/Questions

Jul 23 2020

42mins

Play

359: Throwaway Browser

Podcast cover
Read more

Throw-Away Browser on FreeBSD With "pot" within 5 minutes, OmniOS as OpenBSD guest with bhyve, BSD vs Linux distro development, My FreeBSD Laptop Build, FreeBSD CURRENT Binary Upgrades, and more.

NOTES
This episode of BSDNow is brought to you by Tarsnap

Headlines

Throw-Away Browser on FreeBSD With "pot" Within 5 Minutes

pot is a great and relatively new jail management tool. It offers DevOps style provisioning and can even be used to provide Docker-like, scalable cloud services together with nomad and consul (more about this in Orchestrating jails with nomad and pot).

OpenBSD guest with bhyve - OmniOS

Today I will be creating a OpenBSD guest via bhyve on OmniOS. I will also be adding a Pass Through Ethernet Controller so I can have a multi-homed guest that will serve as a firewall/router.
This post will cover setting up bhyve on OmniOS, so it will also be a good introduction to bhyve. As well, I look into OpenBSD’s uEFI boot loader so if you have had trouble with this, then you are in the right place.

News Roundup

BSD versus Linux distribution development

Q: Comparing-apples-to-BSDs asks: I was reading one of the old articles from the archive. One of the things mentioned was how the BSDs have a distinct approach in terms of packaging the base system relative to userland apps, and that the Linux distros at the time were not following the same practice. Are there Linux distros that have adopted the same approach in modern times? If not, are there technical limitations that are preventing them from doing so, such as some distros supporting multiple kernel versions maybe?
DistroWatch answers: In the article mentioned above, I made the observation that Linux distributions tend to take one of two approaches when it comes to packaging software. Generally a Linux distribution will either offer a rolling release, where virtually all packages are regularly upgraded to their latest stable releases, or a fixed release where almost all packages are kept at a set version number and only receive bug fixes for the life cycle of the distribution. Projects like Arch Linux and Void are popular examples of rolling, always-up-to-date distributions while Fedora and Ubuntu offer fixed platforms.

My FreeBSD Laptop Build

I have always liked Thinkpad hardware and when I started to do more commuting I decided I needed something that had a decent sized screen but fit well on a bus. Luckily about this time Lenovo gave me a nice gift in the Thinkpad X390. Its basically the famous X2xx series but with a 13” screen and smaller bezel.
So with this laptop I figured it was time to actually put the docs together on how I got my FreeBSD workstation working on it. I will here in the near future have another post that will cover this for HardenedBSD as well since the steps are similar but have a few extra gotchas due to the extra hardening.

FreeBSD CURRENT Binary Upgrades

  • Disclaimer This proof-of-concept is not a publication of FreeBSD.
  • Description up.bsd.lv is a proof-of-concept of binary updates for FreeBSD/amd64 CURRENT/HEAD to facilitate the exhaustive testing of FreeBSD and the bhyve hypervisor and OpenZFS 2.0 specifically. Updates are based on the SVN revisions of official FreeBSD Release Engineering bi-monthly snapshots.

Tarsnap

  • This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.

Feedback/Questions

Jul 16 2020

43mins

Play

358: OpenBSD Kubernetes Clusters

Podcast cover
Read more

Yubikey-agent on FreeBSD, Managing Kubernetes clusters from OpenBSD, History of FreeBSD part 1, Running Jitsi-Meet in a FreeBSD Jail, Command Line Bug Hunting in FreeBSD, Game of Github, Wireguard official merged into OpenBSD, and more

NOTES
This episode of BSDNow is brought to you by Tarsnap

Headlines

yubikey-agent on FreeBSD

Some time ago Filippo Valsorda wrote yubikey-agent, seamless SSH agent for YubiKeys. I really like YubiKeys and worked on the FreeBSD support for U2F in Chromium and pyu2f, getting yubikey-agent ported looked like an interesting project. It took some hacking to make it work but overall it wasn’t hard. Following is the roadmap on how to get it set up on FreeBSD. The actual details depend on your system (as you will see)

Manage Kubernetes clusters from OpenBSD

This should work with OpenBSD 6.7. I write this while the source tree is locked for release, so even if I use -current this is as close as -current gets to -release
Update 2020-06-05: we now have a port for kubectl. So, at least in -current things get a bit easier.

News Roundup

History of FreeBSD Part 1: Unix and BSD

FreeBSD, a free and open-source Unix-like operating system has been around since 1993. However, its origins are directly linked to that of BSD, and further back, those of Unix. During this History of FreeBSD series, we will talk about how Unix came to be, and how Berkeley’s Unix developed at Bell Labs.

Running Jitsi-Meet in a FreeBSD Jail

Due to the situation with COVID-19 that also lead to people being confined to their homes in South Africa as well, we decided to provide a (freely usable of course) Jitsi Meet instance to the community being hosted in South Africa on our FreeBSD environment.
That way, communities in South Africa and beyond have a free alternative to the commercial conferencing solutions with sometimes dubious security and privacy histories and at the same time improved user experience due to the lower latency of local hosting.

Command Line Bug Hunting in FreeBSD

FreeBSD uses bugzilla for tracking bugs, taking feature requests, regressions and issues in the Operating System. The web interface for bugzilla is okay, but if you want to do a lot of batch operations it is slow to deal with. We are planning to run a bugsquash on July 11th and that really needs some tooling to help any hackers that show up process the giant bug list we have.

Beastie Bits

Tarsnap

  • This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.

Feedback/Questions

Jul 09 2020

43mins

Play

357: Study the Code

Podcast cover
Read more

OpenBSD 6.7 on PC Engines, NetBSD code study, DRM Update on OpenBSD, Booting FreeBSD on HPE Microserver SATA port, 3 ways to multiboot, and more.

NOTES
This episode of BSDNow is brought to you by Tarsnap

Headlines

OpenBSD 6.7 on PC Engines APU4D4

I just got myself a PC Engines APU4D4. I miss an OpenBSD box providing home services. It’s quite simple to install and run OpenBSD on this machine. And you can even update the BIOS from OpenBSD.

NetBSD code study

News Roundup

Booting FreeBSD off the HPE MicroServer Gen8 ODD SATA port

My small homelab post generated a ton of questions and comments, most of them specific to running FreeBSD on the HP MicroServer. I’ll try and answer these over the coming week.
Josh Paxton emailed to ask how I got FreeBSD booting on it, given the unconventional booting limitations of the hardware. I thought I wrote about it a few years ago, but maybe it’s on my proverbial draft heap. If you’re impatient, the script is in my lunchbox.

3 ways to multiboot

multiboot installation of a BSD system with other operating systems
(OSs) on UEFI hardware is not officially supported by any of the
popular

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv ***

Jul 02 2020

37mins

Play

356: Dig in Deeper

Podcast cover
Read more

TrueNAS is Multi-OS, Encrypted ZFS on NetBSD, FreeBSD’s new Code of Conduct, Gaming on OpenBSD, dig a little deeper, Hammer2 and periodic snapshots, and more.

NOTES
This episode of BSDNow is brought to you by Tarsnap

Headlines

TrueNAS is Multi-OS

There was a time in history where all that mattered was an Operating System (OS) and the hardware it ran on — the “pre-software era”, if you will. Your hardware dictated the OS you used.
Once software applications became prominent, your hardware’s OS determined the applications you could run. Application vendors were forced to juggle the burden of “portability” between OS platforms, choosing carefully the operating systems they’d develop their software to. Then, there were the great OS Wars of the 1990s, replete with the rampant competition, licensing battles, and nasty lawsuits, which more or less gave birth to the “open source OS” era.
The advent of the hypervisor simultaneously gave way to the “virtual era” which set us on a path of agnosticism toward the OS. Instead of choosing from the applications available for your chosen OS, you could simply install another OS on the same hardware for your chosen application. The OS became nothing but a necessary cog in the stack.
TrueNAS open storage enables this “post-OS era” with support for storage clients of all UNIX flavors, Linux, FreeBSD, Windows, MacOS, VMware, Citrix, and many others. Containerization has carried that mentality even further. An operating system, like the hardware that runs it, is now just thought of as part of the “infrastructure”.

Encrypted ZFS on NetBSD 9.0, for a FreeBSD guy

I had one of my other HP Microservers brought back from the office last week to help with this working-from-home world we’re in right now. I was going to wipe an old version of Debian Wheezy/Xen and install FreeBSD to mirror my other machines before thinking: why not NetBSD?

News Roundup

FreeBSD's New Code of Conduct

Gaming on OpenBSD

While no one would expect this, there are huge efforts from a small team to bring more games into OpenBSD. In fact, now some commercial games works natively now, thanks to Mono or Java. There are no wine or linux emulation layer in OpenBSD.
Here is a small list of most well known games that run on OpenBSD:

'dig' a little deeper

I knew the existence of the dig command but didn't exactly know when and how to use it. Then, just recently I encountered an issue that allowed me to learn and make use of it.

HAMMER2 and periodic snapshots

The first version of HAMMER took automatic snapshots, set within the config for each filesystem. HAMMER2 now also takes automatic snapshots, via periodic(8) like most every repeating task on your DragonFly system.

Tarsnap

  • This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.

Feedback/Questions

Jun 25 2020

32mins

Play

355: Man Page Origins

Podcast cover
Read more

Upgrading OpenBSD, Where do Unix man pages come from?, Help for NetBSD’s VAX port, FreeBSD on Dell Latitude 7390, PFS Tool changes in DragonflyBSD, and more.

NOTES
This episode of BSDNow is brought to you by Tarsnap

Headlines

How to Upgrade OpenBSD and Build a Kernel

Let's see how to upgrade your OpenBSD system. Maybe you are doing this because the latest release just came out. If so, this is pretty simple: back up your data, boot from install media, and select "Upgrade" instead of "Install". But maybe the latest release has been out for a few months. Why would we go through the trouble of building and installing a new kernel or other core system components? Maybe some patches have been released to improve system security or stability. It is pretty easy to build and install a kernel on OpenBSD, easier and simpler in many ways than it is on Linux.

The History of man pages

Where do UNIX manpages come from? Who introduced the section-based layout of NAME, SYNOPSIS, and so on? And for manpage authors: where were those economical two- and three-letter instructions developed?

VAX port needs help

The VAX is the oldest machine architecture still supported by NetBSD.
Unfortunately there is another challenge, totally outside of NetBSD, but affecting the VAX port big time: the compiler support for VAX is ... let's say sub-optimal. It is also risking to be dropped completely by gcc upstream.
Now here is where people can help: there is a bounty campaign to finance a gcc hacker to fix the hardest and most immediate issue with gcc for VAX. Without this being resolved, gcc will drop support for VAX in a near future version.

My new FreeBSD Laptop: Dell Latitude 7390

As a FreeBSD developer, I make a point of using FreeBSD whenever I can — including on the desktop. I've been running FreeBSD on laptops since 2004; this hasn't always been easy, but over the years I've found that the situation has generally been improving. One of the things we still lack is adequate documentation, however — so I'm writing this to provide an example for users and also Google bait in case anyone runs into some of the problems I had to address.

PFS tool changes in DragonFly

HAMMER2 just became a little more DWIM: the pfs-list and pfs-delete directives will now look across all mounted filesystems, not just the current directory’s mount path. pfs-delete won’t delete any filesystem name that appears in more than one place, though

  • git: hammer2 - Enhance pfs-list and pfs-delete Enhance pfs-list to list PFSs available across all mounted hammer2 filesystems instead of just the current directory's mount. A specific mount may be specified via -s mountpt. Enhance pfs-delete to look for the PFS name across all mounted hammer2 filesystems instead of just the current directory's mount. As a safety, pfs-delete will refuse to delete PFS names which are duplicated across multiple mounts. A specific mount may be specified via -s mountpt.

Beastie Bits

Feedback/Questions

Jun 18 2020

40mins

Play

354: ZFS safekeeps data

Podcast cover
Read more

FreeBSD 11.4-RC 2 available, OpenBSD 6.7 on a PineBook Pro 64, How OpenZFS Keeps Your Data Safe, Bringing FreeBSD to EC2, FreeBSD 2020 Community Survey, and more.

NOTES
This episode of BSDNow is brought to you by Tarsnap

Headlines

FreeBSD 11.4-RC2 Now Available

The second RC build of the 11.4-RELEASE release cycle is now available.

Install OpenBSD 6.7-current on a PineBook Pro 64

This document is work in progress and I'll update the date above once I change something. If you have something to add, remarks, etc please contact me. Preferably via Mastodon but other means of communication are also fine.

News Roundup

Understanding How OpenZFS Keeps Your Data Safe

Veteran technology writer Jim Salter wrote an excellent guide on the ZFS file system’s features and performance that we absolutely had to share. There’s plenty of information in the article for ZFS newbies and advanced users alike. Be sure to check out the article over at Ars Technica to learn more about ZFS concepts including pools, vdevs, datasets, snapshots, and replication, just to name a few.

Bringing FreeBSD to ec2

Colin is the founder of Tarsnap, a secure online backup service which combines the flexibility and scriptability of the standard UNIX "tar" utility with strong encryption, deduplication, and the reliability of Amazon S3 storage. Having started work on Tarsnap in 2006, Colin is among the first generation of users of Amazon Web Services, and has written dozens of articles about his experiences with AWS on his blog.

FreeBSD 2020 Community Survey

The FreeBSD Core Team invites you to complete the 2020 FreeBSD Community Survey. The purpose of this survey is to collect quantitative data from the public in order to help guide the project’s priorities and efforts. This is only the second time a survey has been conducted by the FreeBSD Project and your input is valued.
The survey will remain open for 14 days and will close on June 16th at 17:00 UTC (Tuesday 10am PDT).

Beastie Bits

Feedback/Questions

Sponsored By:

Jun 11 2020

35mins

Play

353: ZFS on Ironwolf

Podcast cover
Read more

Scheduling in NetBSD, ZFS vs. RAID on Ironwolf disks, OpenBSD on Microsoft Surface Go 2, FreeBSD for Linux sysadmins, FreeBSD on Lenovo T480, and more.

NOTES
This episode of BSDNow is brought to you by Tarsnap

Headlines

Scheduling in NetBSD – Part 1

In this blog, we will discuss about the 4.4BSD Thread scheduler one of the two schedulers in NetBSD and a few OS APIs that can be used to control the schedulers and get information while executing.

ZFS versus RAID: Eight Ironwolf disks, two filesystems, one winner

This has been a long while in the making—it's test results time. To truly understand the fundamentals of computer storage, it's important to explore the impact of various conventional RAID (Redundant Array of Inexpensive Disks) topologies on performance. It's also important to understand what ZFS is and how it works. But at some point, people (particularly computer enthusiasts on the Internet) want numbers.

  • If you want to hear more from Jim, he has a new bi-weekly podcast with Allan and Joe Ressington over at 2.5admins.com

News Roundup

OpenBSD on the Microsoft Surface Go 2

I used OpenBSD on the original Surface Go back in 2018 and many things worked with the big exception of the internal Atheros WiFi. This meant I had to keep it tethered to a USB-C dock for Ethernet or use a small USB-A WiFi dongle plugged into a less-than-small USB-A-to-USB-C adapter.

FreeBSD UNIX for Linux sysadmins

If you’ve ever installed and explored another Linux distro (what Linux sysadmin hasn’t?!?), then exploring FreeBSD is going be somewhat similar with a few key differences.
While there is no graphical installation, the installation process is straightforward and similar to installing a server-based Linux distro. Just make sure you choose the local_unbound package when prompted if you want to cache DNS lookups locally, as FreeBSD doesn’t have a built-in local DNS resolver that does this.
Following installation, the directory structure is almost identical to Linux. Of course, you’ll notice some small differences here and there (e.g. regular user home directories are located under /usr/home instead of /home). Standard UNIX commands such as ls, chmod, find, which, ps, nice, ifconfig, netstat, sockstat (the ss command in Linux) are exactly as you’d expect, but with some different options here and there that you’ll see in the man pages. And yes, reboot and poweroff are there too.

FreeBSD on the Lenovo Thinkpad T480

Recently I replaced my 2014 MacBook Air with a Lenovo Thinkpad T480, on which I've installed FreeBSD, currently 12.1-RELEASE. This page documents my set-up along with various configuration tweaks and fixes.

Tarsnap

  • This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Sponsored By:

Jun 04 2020

38mins

Play

352: Introducing Randomness

Podcast cover
Read more

A brief introduction to randomness, logs grinding netatalk to a halt, NetBSD core team changes, Using qemu guest agent on OpenBSD kvm/qemu guests, WireGuard patchset for OpenBSD, FreeBSD 12.1 on a laptop, and more.

Headlines

Entropy

A brief introduction to randomness

  • Problem: Computers are very predictable. This is by design.

But what if we want them to act unpredictably? This is very useful if we want to secure our private communications with randomized keys, or not let people cheat at video games, or if we're doing statistical simulations or similar.

Logs grinding Netatalk on FreeBSD to a hault

I’ve heard it said the cobbler’s children walk barefoot. While posessing the qualities of a famed financial investment strategy, it speaks to how we generally put more effort into things for others than ourselves; at least in business.
The HP Microserver I share with Clara is a modest affair compared to what we run at work. It has six spinning rust drives and two SSDs which are ZFS-mirrored; not even in a RAID 10 equivalent. This is underlaid with GELI for encryption, and served to our Macs with Netatalk over gigabit Ethernet with jumbo frames.

News Roundup

NetBSD Core Team Changes

Matt Thomas (matt@) has served on the NetBSD core team for over ten years, and has made many contributions, including ELF functionality, being the long-time VAX maintainer, gcc contributor, the generic pmap, and also networking functionality, and platform bring-up over the years. Matt has stepped down from the NetBSD core team, and we thank him for his many, extensive contributions.
Robert Elz (kre@), a long time BSD contributor, has kindly accepted the offer to join the core team, and help us out with the benefit of his experience and advice over many years. Amongst other things, Robert has been maintaining our shell, liaising with the Austin Group, and bringing it up to date with modern functionality.

Using qemu guest agent on OpenBSD kvm/qemu guests

In a post to the ports@ mailing list, Landry Breuil (landry@) shared some of his notes on using qemu guest agent on OpenBSD kvm/qemu guests.

WireGuard patchset for OpenBSD

A while ago I wanted to learn more about OpenBSD development. So I picked a project, in this case WireGuard, to develop a native client for. Over the last two years, with many different iterations, and working closely with the WireGuard's creator (Jason [Jason A. Donenfeld - Ed.], CC'd), it started to become a serious project eventually reaching parity with other official implementations. Finally, we are here and I think it is time for any further development to happen inside the src tree.

FreeBSD 12.1 on a laptop

I’m using FreeBSD again on a laptop for some reasons so expect to read more about FreeBSD here. This tutorial explain how to get a graphical desktop using FreeBSD 12.1.

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

May 28 2020

50mins

Play

351: Heaven: OpenBSD 6.7

Podcast cover
Read more

Backup and Restore on NetBSD, OpenBSD 6.7 available, Building a WireGuard Jail with FreeBSD's standard tools, who gets to chown things and quotas, influence TrueNAS CORE roadmap, and more.

Headlines

Backup and Restore on NetBSD

Putting together the bits and pieces of a backup and restore concept, while not being rocket science, always seems to be a little bit ungrateful. Most Admin Handbooks handle this topic only within few pages. After replacing my old Mac Mini's OS by NetBSD, I tried to implement an automated backup, allowing me to handle it similarly to the time machine backups I've been using before. Suggestions on how to improve are always welcome.

BSD Release: OpenBSD 6.7

The OpenBSD project produces and operating system which places focus on portability, standardisation, code correctness, proactive security and integrated cryptography. The project's latest release is OpenBSD 6.7 which introduces several new improvements to the cron scheduling daemon, improvements to the web server daemon, and the top command now offers scrollable output. These and many more changes can be found in the project's release announcement: "This is a partial list of new features and systems included in OpenBSD 6.7. For a comprehensive list, see the changelog leading to 6.7. General improvements and bugfixes: Reduced the minimum allowed number of chunks in a CONCAT volume from 2 to 1, increasing the number of volumes which can be created on a single disk with bioctl(8) from 7 to 15. This can be used to create more partitions than previously. Rewrote the cron(8) flag-parsing code to be getopt-like, allowing tight formations like -ns and flag repetition. Renamed the 'options' field in crontab(5) to 'flags'. Added crontab(5) -s flag to the command field, indicating that only a single instance of the job should run concurrently. Added cron(8) support for random time values using the ~ operator. Allowed cwm(1) configuration of window size based on percentage of the master window during horizontal and vertical tiling actions."

News Roundup

Building a WireGuard Jail with the FreeBSD's Standard Tools

Recently, I had an opportunity to build a WireGuard jail on a FreeBSD 12.1 host.
As it was really quick and easy to setup and it has been working completely fine for a month, I’d like to share my experience with anyone interested in this topic.

The Unix divide over who gets to chown things, and (disk space) quotas

One of the famous big splits between the BSD Unix world and the System V world is whether ordinary users can use chown (the command and the system call) to give away their own files. In System V derived Unixes you were generally allowed to; in BSD derived Unixes you weren't. Until I looked it up now to make sure, I thought that BSD changed this behavior from V7 and that V7 had an unrestricted chown. However, this turns out to be wrong; in V7 Unix, chown(2) was restricted to root only.

You Can Influence the TrueNAS CORE Roadmap!

As many of you know, we’ve historically had three ticket types available in our tracker: Bugs, Features, and Improvements, which are all fairly self-explanatory. After some discussion internally, we’ve decided to implement a new type of ticket, a “Suggestion”. These will be replacing Feature and Improvement requests for the TrueNAS Community, simplifying things down to two options: Bugs and Suggestions. This change also introduces a slightly different workflow than before.

Beastie Bits

Feedback/Questions

May 21 2020

49mins

Play

350: Speedy Bridges

Podcast cover
Read more

5x if_bridge Performance Improvement, How Unix Won, Understanding VLAN Configuration on FreeBSD, Using bhyve PCI passthrough on OmniOS, TrueNAS 11.3-U2 Available, and more.

Headlines

5x if_bridge Performance Improvement

With FreeBSD Foundation grant, Kristof Provost harnesses new parallel techniques to uncork performance bottleneck

How Unix Won

+> Unix has won in every conceivable way. And in true mythic style, it contains the seeds of its own eclipse. This is my subjective historical narrative of how that happened.

I’m using the name “Unix” to include the entire family of operating systems descended from it, or that have been heavily influenced by it. That includes Linux, SunOS, Solaris, BSD, Mac OS X, and many, many others.
Both major mobile OSs, Android and iOS, have Unix roots. Their billions of users dwarf those using clunky things like laptops and desktops, but even there, Windows is only the non-Unix viable OS. Almost everything running server-side in giant datacenters is Linux.
How did Unix win?

News Roundup

Check logs of central syslog-ng log host on FreeBSD

This blog post continues where the blog post A central log host with syslog-ng on FreeBSD left off. Open source solutions to check syslog log messages exist, such as Logcheck or Logwatch. Although these are not to difficult to implement and maintain, I still found these to much. So I went for my own home grown solution to check the syslog messages of the SoCruel.NU central log host. And the solution presented in this blog post works pretty well for me!

Understanding VLAN Configuration on FreeBSD

Until recently, I’ve never had a chance to use VLANs on FreeBSD hosts, though I sometimes configure them on ethernet switches.
But when I was playing with vnet jails, I suddenly got interested in VLAN configuration on FreeBSD and experimented with it for some time.
I wrote this short article to summarize my current understanding of how to configure VLANs on FreeBSD.

Using bhyve PCI passthrough on OmniOS

Some hardware is not supported in illumos yet, but luckily there is bhyve which supports pci passthrough to any guest operating system. To continue with my OmniOS desktop on "modern" hardware I would love wifi support, so why not using a bhyve guest as router zone which provide the required drivers?

TrueNAS 11.3-U2 is Generally Available

TrueNAS 11.3-U2.1 is generally available as of 4/22/2020. This update is based on FreeNAS 11.3-U2 which has had over 50k deployments and received excellent community and third party reviews. The Release Notes are available on the iXsystems.com website.

Beastie Bits

HardenedBSD April 2020 Status Report
NYC Bug’s Mailing List - Listing of open Dev Jobs

Feedback/Questions

May 14 2020

34mins

Play

349: Entropy Overhaul

Podcast cover
Read more

Encrypted Crash Dumps in FreeBSD, Time on Unix, Improve ZVOL sync write performance with a taskq, central log host with syslog-ng, NetBSD Entropy overhaul, Setting Up NetBSD Kernel Dev Environment, and more.

Headlines

EKCD - Encrypted Crash Dumps in FreeBSD

Some time ago, I was describing how to configure networking crash dumps. In that post, I mentioned that there is also the possibility to encrypt crash dumps. Today we will look into this functionality. Initially, it was implemented during Google Summer of Code 2013 by my friend Konrad Witaszczyk, who made it available in FreeBSD 12. If you can understand Polish, you can also look into his presentation on BSD-PL on which he gave a comprehensive review of all kernel crash dumps features.

The main issue with crash dumps is that they may include sensitive information available in memory during a crash. They will contain all the data from the kernel and the userland, like passwords, private keys, etc. While dumping them, they are written to unencrypted storage, so if somebody took out the hard drive, they could access sensitive data. If you are sending a crash dump through the network, it may be captured by third parties. Locally the data are written directly to a dump device, skipping the GEOM subsystem. The purpose of that is to allow a kernel to write a crash dump even in case a panic occurs in the GEOM subsystem. It means that a crash dump cannot be automatically encrypted with GELI.

Time on Unix

Time, a word that is entangled in everything in our lives, something we’re intimately familiar with. Keeping track of it is important for many activities we do.

Over millennia we’ve developed different ways to calculate it. Most prominently, we’ve relied on the position the sun appears to be at in the sky, what is called apparent solar time.

We’ve decided to split it as seasons pass, counting one full cycle of the 4 seasons as a year, a full rotation around the sun. We’ve also divided the passing of light to the lack thereof as days, a rotation of the earth on itself. Moving on to more precise clock divisions such as seconds, minutes, and hours, units that meant different things at different points in history. Ultimately, as travel got faster, the different ways of counting time that evolved in multiple places had to converge. People had to agree on what it all meant.

See the article for more

News Roundup

Improve ZVOL sync write performance by using a taskq

A central log host with syslog-ng on FreeBSD - Part 1

syslog-ng is the Swiss army knife of log management. You can collect logs from any source, process them in real time and deliver them to wide range of destinations. It allows you to flexibly collect, parse, classify, rewrite and correlate logs from across your infrastructure. This is why syslog-ng is the perfect solution for the central log host of my (mainly) FreeBSD based infrastructure.

HEADS UP: NetBSD Entropy Overhaul

This week I committed an overhaul of the kernel entropy system. Please let me know if you observe any snags! For the technical background, see the thread on tech-kern a few months ago: https://mail-index.NetBSD.org/tech-kern/2019/12/21/msg025876.html.

Setting Up NetBSD Kernel Dev Environment

I used T_PAGEFLT’s blog post as a reference for setting my NetBSD kernel development environment since his website is down I’m putting down the steps here so it would be helpful for starters.

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

May 07 2020

57mins

Play

348: BSD Community Collections

Podcast cover
Read more

FuryBSD 2020Q2 Images Available, Technical reasons to choose FreeBSD over GNU/Linux, Ars technica reviews GhostBSD, “TLS Mastery” sponsorships open, BSD community show their various collections, a tale of OpenBSD secure memory allocator internals, learn to stop worrying and love SSDs, and more.

Headlines

FuryBSD 2020Q2 Images Available for XFCE and KDE

The Q2 2020 images are not a visible leap forward but a functional leap forward. Most effort was spent creating a better out of box experience for automatic Ethernet configuration, working WiFi, webcam, and improved hypervisor support.

Technical reasons to choose FreeBSD over GNU/Linux

Since I wrote my article "Why you should migrate everything from Linux to BSD" I have been wanting to write something about the technical reasons to choose FreeBSD over GNU/Linux and while I cannot possibly cover every single reason, I can write about some of the things that I consider worth noting.

News Roundup

+ Not actually Linux distro review deux: GhostBSD

When I began work on the FreeBSD 12.1-RELEASE review last week, it didn't take long to figure out that the desktop portion wasn't going very smoothly.

I think it's important for BSD-curious users to know of easier, gentler alternatives, so I did a little looking around and settled on GhostBSD for a follow-up review.

GhostBSD is based on TrueOS, which itself derives from FreeBSD Stable. It was originally a Canadian distro, but—like most successful distributions—it has transcended its country of origin and can now be considered worldwide. Significant GhostBSD development takes place now in Canada, Italy, Germany, and the United States.

“TLS Mastery” sponsorships open

My next book will be TLS Mastery, all about Transport Layer Encryption, Let’s Encrypt, OCSP, and so on.

This should be a shorter book, more like my DNSSEC or Tarsnap titles, or the first edition of Sudo Mastery. I would like a break from writing doorstops like the SNMP and jails books.

JT (our producer) shared his Open Source Retail Box Collection on twitter this past weekend and there was a nice response from a few in the BSD Community showing their collections:

Do you have a nice collection, take a picture and send it in!

Tale of OpenBSD secure memory allocator internals - malloc(3)

Hi there,

It's been a very long time I haven't written anything after my last OpenBSD blogs, that is,

OpenBSD Kernel Internals — Creation of process from user-space to kernel space.

OpenBSD: Introduction to execpromises in the pledge(2)

pledge(2): OpenBSD's defensive approach to OS Security

So, again I started reading OpenBSD source codes with debugger after reducing my sleep timings and managing to get some time after professional life. This time I have picked one of my favourite item from my wishlist to learn and share, that is, OpenBSD malloc(3), secure allocator

How I learned to stop worrying and love SSDs

my home FreeNAS runs two pools for data. One RAIDZ2 with four spinning disk drives and one mirror with two SSDs. Toying with InfluxDB and Grafana in the last couple of days I found that I seem to have a constant write load of 1 Megabyte (!) per second on the SSDs. What the ...?

So I run three VMs on the SSDs in total. One with Windows 10, two with Ubuntu running Confluence, A wiki essentially, with files for attachments and MySQL as the backend database. Clearly the writes had to stop when the wikis were not used at all, just sitting idle, right?

Well even with a full query log and quite some experience in the operation of web applications I could not figure out what Confluence is doing (productively, no doubt) but trust me, it writes a couple of hundred kbytes to the database each second just sitting idle.

My infrastructure as of 2019

I've wanted to write about my infrastructure for a while, but I kept thinking, "I'll wait until after I've done $next_thing_on_my_todo." Of course this cycle never ends, so I decided to write about its state at the end of 2019. Maybe I'll write an update on it in a couple of moons; who knows?

For something different than our usual Beastie Bits… we bring you…

We're all quarantined so lets install BSD on things! Install BSD on something this week, write it up and let us know about it, and maybe we'll feature you!

BSDNow is going Independent

  • After being part of Jupiter Broadcasting since we started back in 2013, BSDNow is moving to become independent. We extend a very large thank you to Jupiter Broadcasting and Linux Academy for hosting us for so many years, and allowing us to bring you over 100 episodes without advertisements.
    What does this mean for you, the listener? Not much will change, just make sure your subscription is via the RSS feed at BSDNow.tv rather than one of the Jupiter Broadcasting feeds. We will update you with more news as things settle out.

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

Apr 30 2020

1hr

Play

347: New Directions

Podcast cover
Read more

Rethinking OpenBSD security, FreeBSD 2020 Q1 status report, the notion of progress and user interfaces, Comments about Thomas E. Dickey on NetBSD curses, making Unix a little more Plan9-like, Not-actually Linux distro review: FreeBSD, and more.

Headlines

Rethinking OpenBSD Security

OpenBSD aims to be a secure operating system. In the past few months there were quite a few security errata, however. That’s not too unusual, but some of the recent ones were a bit special. One might even say bad. The OpenBSD approach to security has a few aspects, two of which might be avoiding errors and minimizing the risk of mistakes. Other people have other ideas about how to build secure systems. I think it’s worth examining whether the OpenBSD approach works, or if this is evidence that it’s doomed to failure.
I picked a few errata, not all of them, that were interesting and happened to suit my narrative.

FreeBSD 2020 Q1 Quarterly report

Welcome, to the quarterly reports, of the future! Well, at least the first quarterly report from 2020. The new timeline, mentioned in the last few reports, still holds, which brings us to this report, which covers the period of January 2020 - March 2020.

News Roundup

The Notion of Progress and User Interfaces

One trait of modern Western culture is the notion of progress. A view claiming, at large, everything is getting better and better.

How should we think about progress? Both in general and regarding technology?

Thomas E. Dickey on NetBSD curses

I was recently pointed at a web page on Thomas E. Dickeys site talking about NetBSD curses. It seems initially that the page was intended to be a pointer to some differences between ncurses and NetBSD curses and does appear to start off in this vein but it seems that the author has lost the plot as the document evolved and the tail end of it seems to be devolving into some sort of slanging match. I don't want to go through Mr. Dickey's document point by point, that would be tedious but I would like to pick out some of the things that I believe to be the most egregious. Please note that even though I am a NetBSD developer, the opinions below are my own and not the NetBSD projects.

Making Unix a little more Plan9-like

I’m not really interested in defending anything. I tried out plan9port and liked it, but I have to live in Unix land. Here’s how I set that up.

A Warning

The suckless community, and some of the plan9 communities, are dominated by jackasses. I hope that’s strong enough wording to impress the severity. Don’t go into IRC for help. Stay off the suckless email list. The software is great, the people who write it are well-spoken and well-reasoned, but for some reason the fandom is horrible to everyone.

Not-actually Linux distro review: FreeBSD 12.1-RELEASE

This month's Linux distro review isn't of a Linux distribution at all—instead, we're taking a look at FreeBSD, the original gangster of free Unix-like operating systems.

The first FreeBSD release was in 1993, but the operating system's roots go further back—considerably further back. FreeBSD started out in 1992 as a patch-release of Bill and Lynne Jolitz's 386BSD—but 386BSD itself came from the original Berkeley Software Distribution (BSD). BSD itself goes back to 1977—for reference, Linus Torvalds was only seven years old then.

Before we get started, I'd like to acknowledge something up front—our distro reviews include the desktop experience, and that is very much not FreeBSD's strength. FreeBSD is far, far better suited to running as a headless server than as a desktop! We're going to get a full desktop running on it anyway, because according to Lee Hutchinson, I hate myself—and also because we can't imagine readers wouldn't care about it.

FreeBSD does not provide a good desktop experience, to say the least. But if you're hankering for a BSD-based desktop, don't worry—we're already planning a followup review of GhostBSD, a desktop-focused BSD distribution.

Beastie Bits

BSDNow is going Independent

  • After being part of Jupiter Broadcasting since we started back in 2013, BSDNow is moving to become independent. We extend a very large thank you to Jupiter Broadcasting and Linux Academy for hosting us for so many years, and allowing us to bring you over 100 episodes without advertisements. LinuxAcademy is now under new leadership, and we understand that cutbacks needed to be made, and that BSD is not their core product. That does not mean your favourite BSD podcast is going away, we will continue and we expect things will not look much different. What does this mean for you, the listener? Not much will change, just make sure your subscription is via the RSS feed at BSDNow.tv rather than one of the Jupiter Broadcasting feeds. We will update you with more news as things settle out.

Feedback/Questions

Your browser does not support the HTML5 video tag.

Apr 23 2020

1hr

Play

346: Core File Tales

Podcast cover
Read more

Tales from a core file, Lenovo X260 BIOS Update with OpenBSD, the problem of Unix iowait and multi-CPU machines, Hugo workflow using FreeBSD Jails, Caddy, Restic; extending NetBSD-7 branch support, a tale of two hypervisor bugs, and more.

Headlines

Tales From a Core File - Lessons from the Unix stdio ABI: 40 Years Later

On the side, I’ve been wrapping up some improvements to the classic Unix stdio libraries in illumos. stdio contains the classic functions like fopen(), printf(), and the security nightmare gets(). While working on support for fmemopen() and friends I got to reacquaint myself with some of the joys of the stdio ABI and its history from 7th Edition Unix. With that in mind, let’s dive into this, history, and some mistakes not to repeat. While this is written from the perspective of the C programming language, aspects of it apply to many other languages.

Update Lenovo X260 BIOS with OpenBSD

My X260 only runs OpenBSD and has no CD driver. But I still need to upgrade its BIOS from time to time. And this is possible using the ISO BIOS image.

First off all, you need to download the “BIOS Update (Bootable CD)” from the Lenovo Support Website.

News Roundup

The problem of Unix iowait and multi-CPU machines

Various Unixes have had a 'iowait' statistic for a long time now (although I can't find a source for where it originated; it's not in 4.x BSD, so it may have come through System V and sar). The traditional and standard definition of iowait is that it's the amount of time the system was idle but had at least one process waiting on disk IO. Rather than count this time as 'idle' (as you would if you had a three-way division of CPU time between user, system, and idle), some Unixes evolved to count this as a new category, 'iowait'.

My Latest Self Hosted Hugo Workflow using FreeBSD Jails, Caddy, Restic and More

After hosting with Netlify for a few years, I decided to head back to self hosting. Theres a few reasons for that but the main reasoning was that I had more control over how things worked.

In this post, i’ll show you my workflow for deploying my Hugo generated site (www.jaredwolff.com). Instead of using what most people would go for, i’ll be doing all of this using a FreeBSD Jails based server. Plus i’ll show you some tricks i’ve learned over the years on bulk image resizing and more.

Let’s get to it.

Extending support for the NetBSD-7 branch

Typically, some time after releasing a new NetBSD major version (such as NetBSD 9.0), we will announce the end-of-life of the N-2 branch, in this case NetBSD-7.

We've decided to hold off on doing that to ensure our users don't feel rushed to perform a major version update on any remote machines, possibly needing to reach the machine if anything goes wrong.

Security fixes will still be made to the NetBSD-7 branch.

We hope you're all safe. Stay home.

Tale of two hypervisor bugs - Escaping from FreeBSD bhyve

VM escape has become a popular topic of discussion over the last few years. A good amount of research on this topic has been published for various hypervisors like VMware, QEMU, VirtualBox, Xen and Hyper-V. Bhyve is a hypervisor for FreeBSD supporting hardware-assisted virtualization. This paper details the exploitation of two bugs in bhyve - FreeBSD-SA-16:32.bhyve (VGA emulation heap overflow) and CVE-2018-17160 (Firmware Configuration device bss buffer overflow) and some generic techniques which could be used for exploiting other bhyve bugs. Further, the paper also discusses sandbox escapes using PCI device passthrough, and Control-Flow Integrity bypasses in HardenedBSD 12-CURRENT

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

Apr 16 2020

55mins

Play

345: Switchers to BSD

Podcast cover
Read more

NetBSD 8.2 is available, NextCloud on OpenBSD, X11 screen locking, NetBSD and RISC OS running parallel, community feedback about switching to BSD, and more.

Headlines

NetBSD 8.2 is available!

The third release in the NetBSD-8 is now available.

This release includes all the security fixes in NetBSD-8 up until this point, and other fixes deemed important for stability.

  • Some highlights include:
    • x86: fixed regression in booting old CPUs
    • x86: Hyper-V Gen.2 VM framebuffer support
    • httpd(8): fixed various security issues
    • ixg(4): various fixes / improvements
    • x86 efiboot: add tftp support, fix issues on machines with many memory segments, improve graphics mode logic to work on more machines.
    • Various kernel memory info leaks fixes
    • Update expat to 2.2.8
    • Fix ryzen USB issues and support xHCI version 3.10.
    • Accept root device specification as NAME=label.
    • Add multiboot 2 support to x86 bootloaders.
    • Fix for CVE-2019-9506: 'Key Negotiation of Bluetooth' attack.
    • nouveau: limit the supported devices and fix firmware loading.
    • radeon: fix loading of the TAHITI VCE firmware.
    • named(8): stop using obsolete dnssec-lookaside.

NextCloud on OpenBSD

NextCloud and OpenBSD are complementary to one another. NextCloud is an awesome, secure and private alternative for proprietary platforms, whereas OpenBSD forms the most secure and solid foundation to serve it on. Setting it up in the best way isn’t hard, especially using this step by step tutorial.

  • Preface

Back when this tutorial was initially written, things were different. The OpenBSD port relied on PHP 5.6 and there were no package updates. But the port improved (hats off, Gonzalo!) and package updates were introduced to the -stable branch (hats off, Solene!).

A rewrite of this tutorial was long overdue. Right now, it is written for 6.6 -stable and will be updated once 6.7 is released. If you have any questions or desire some help, feel free to reach out.

News Roundup

X11 screen locking: a secure and modular approach

For years I’ve been using XScreenSaver as a default, but I recently learned about xsecurelock and re-evaluated my screen-saving requirements

NetBSD and RISC OS running parallel

I have been experimenting with running two systems at the same time on the RK3399 SoC.
It all begun when I figured out how to switch to the A72 cpu for RISC OS. When the switch was done, the A53 cpu just continued to execute code.
OK I thought why not give it something to do!
My first step was to run some small programs.
It worked!

  • Thanks to Tom Jones for the pointer to this article

Several weeks ago we covered a story about switching from Linux to BSD. Benedict and JT asked for community feedback as to their thoughts on the matter. Allan was out that week, so this will give him an opportunity to chime in with his thoughts as well.

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Your browser does not support the HTML5 video tag.

Apr 09 2020

47mins

Play

344: Grains of Salt

Podcast cover
Read more

Shell text processing, data rebalancing on ZFS mirrors, Add Security Headers with OpenBSD relayd, ZFS filesystem hierarchy in ZFS pools, speeding up ZSH, How Unix pipes work, grow ZFS pools over time, the real reason ifconfig on Linux is deprecated, clear your terminal in style, and more.

Headlines

Text processing in the shell

This article is part of a self-published book project by Balthazar Rouberol and Etienne Brodu, ex-roommates, friends and colleagues, aiming at empowering the up and coming generation of developers. We currently are hard at work on it!

One of the things that makes the shell an invaluable tool is the amount of available text processing commands, and the ability to easily pipe them into each other to build complex text processing workflows. These commands can make it trivial to perform text and data analysis, convert data between different formats, filter lines, etc.

When working with text data, the philosophy is to break any complex problem you have into a set of smaller ones, and to solve each of them with a specialized tool.

Rebalancing data on ZFS mirrors

One of the questions that comes up time and time again about ZFS is “how can I migrate my data to a pool on a few of my disks, then add the rest of the disks afterward?”

If you just want to get the data moved and don’t care about balance, you can just copy the data over, then add the new disks and be done with it. But, it won’t be distributed evenly over the vdevs in your pool.

Don’t fret, though, it’s actually pretty easy to rebalance mirrors. In the following example, we’ll assume you’ve got four disks in a RAID array on an old machine, and two disks available to copy the data to in the short term.

News Roundup

Using OpenBSD relayd to Add Security Headers

I am a huge fan of OpenBSD’s built-in httpd server as it is simple, secure, and quite performant. With the modern push of the large search providers pushing secure websites, it is now important to add security headers to your website or risk having the search results for your website downgraded. Fortunately, it is very easy to do this when you combine httpd with relayd. While relayd is principally designed for layer 3 redirections and layer 7 relays, it just so happens that it makes a handy tool for adding the recommended security headers. My website automatically redirects users from http to https and this gets achieved using a simple redirection in /etc/httpd.conf So if you have a configuration similar to mine, then you will still want to have httpd listen on the egress interface on port 80. The key thing to change here is to have httpd listen on 127.0.0.1 on port 443.

How we set up our ZFS filesystem hierarchy in our ZFS pools

Our long standing practice here, predating even the first generation of our ZFS fileservers, is that we have two main sorts of filesystems, home directories (homedir filesystems) and what we call 'work directory' (workdir) filesystems. Homedir filesystems are called /h/NNN (for some NNN) and workdir filesystems are called /w/NNN; the NNN is unique across all of the different sorts of filesystems. Users are encouraged to put as much stuff as possible in workdirs and can have as many of them as they want, which mattered a lot more in the days when we used Solaris DiskSuite and had fixed-sized filesystems.

Speeding up ZSH

https://web.archive.org/web/20200315184849/https://blog.jonlu.ca/posts/speeding-up-zsh

I was opening multiple shells for an unrelated project today and noticed how abysmal my shell load speed was. After the initial load it was relatively fast, but the actual shell start up was noticeably slow. I timed it with time and these were the results.

In the future I hope to actually recompile zsh with additional profiling techniques and debug information - keeping an internal timer and having a flag output current time for each command in a tree fashion would make building heat maps really easy.

How do Unix Pipes work

Pipes are cool! We saw how handy they are in a previous blog post. Let’s look at a typical way to use the pipe operator. We have some output, and we want to look at the first lines of the output. Let’s download The Brothers Karamazov by Fyodor Dostoevsky, a fairly long novel.

What we do to enable us to grow our ZFS pools over time

In my entry on why ZFS isn't good at growing and reshaping pools, I mentioned that we go to quite some lengths in our ZFS environment to be able to incrementally expand our pools. Today I want to put together all of the pieces of that in one place to discuss what those lengths are.
Our big constraint is that not only do we need to add space to pools over time, but we have a fairly large number of pools and which pools will have space added to them is unpredictable. We need a solution to pool expansion that leaves us with as much flexibility as possible for as long as possible. This pretty much requires being able to expand pools in relatively small increments of space.

Linux maintains bugs: The real reason ifconfig on Linux is deprecated

In my third installment of FreeBSD vs Linux, I will discuss underlying reasons for why Linux moved away from ifconfig(8) to ip(8).

In the past, when people said, “Linux is a kernel, not an operating system”, I knew that was true but I always thought it was a rather pedantic criticism. Of course no one runs just the Linux kernel, you run a distribution of Linux. But after reviewing userland code, I understand the significant drawbacks to developing “just a kernel” in isolation from the rest of the system.

Clear Your Terminal in Style

if you’re someone like me who habitually clears their terminal, sometimes you want a little excitement in your life. Here is a way to do just that.

This post revolves around the idea of giving a command a percent chance of running. While the topic at hand is not serious, this simple technique has potential in your scripts.

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Your browser does not support the HTML5 video tag.

Apr 02 2020

55mins

Play

343: FreeBSD, Corona: Fight!

Podcast cover
Read more

Fighting the Coronavirus with FreeBSD, Wireguard VPN Howto in OPNsense, NomadBSD 1.3.1 available, fresh GhostBSD 20.02, New FuryBSD XFCE and KDE images, pf-badhost 0.3 released, and more.

Headlines

Fighting the Coronavirus with FreeBSD

Here is a quick HOWTO for those who want to provide some FreeBSD based compute resources to help finding vaccines.

UPDATE 2020-03-22: 0mp@ made a port out of this, it is in “biology/linux-foldingathome”.

Per default it will now pick up some SARS-CoV‑2 (COVID-19) related folding tasks. There are some more config options (e.g. how much of the system resources are used). Please refer to the official Folding@Home site for more information about that. Be also aware that there is a big rise in compute resources donated to Folding@Home, so the pool of available work units may be empty from time to time, but they are working on adding more work units. Be patient.

How to configure the Wireguard VPN in OPNsense

WireGuard is a modern designed VPN that uses the latest cryptography for stronger security, is very lightweight, and is relatively easy to set up (mostly). I say ‘mostly’ because I found setting up WireGuard in OPNsense to be more difficult than I anticipated. The basic setup of the WireGuard VPN itself was as easy as the authors claim on their website, but I came across a few gotcha's. The gotcha's occur with functionality that is beyond the scope of the WireGuard protocol so I cannot fault them for that. My greatest struggle was configuring WireGuard to function similarly to my OpenVPN server. I want the ability to connect remotely to my home network from my iPhone or iPad, tunnel all traffic through the VPN, have access to certain devices and services on my network, and have the VPN devices use my home's Internet connection.

WireGuard behaves more like a SSH server than a typical VPN server. With WireGuard, devices which have shared their cryptographic keys with each other are able to connect via an encrypted tunnel (like a SSH server configured to use keys instead of passwords). The devices that are connecting to one another are referred to as “peer” devices. When the peer device is an OPNsense router with WireGuard installed, for instance, it can be configured to allow access to various resources on your network. It becomes a tunnel into your network similar to OpenVPN (with the appropriate firewall rules enabled). I will refer to the WireGuard installation on OPNsense as the server rather than a “peer” to make it more clear which device I am configuring unless I am describing the user interface because that is the terminology used interchangeably by WireGuard.

The documentation I found on WireGuard in OPNsense is straightforward and relatively easy to understand, but I had to wrestle with it for a little while to gain a better understanding on how it should be configured. I believe it was partially due to differing end goals – I was trying to achieve something a little different than the authors of other wiki/blog/forum posts. Piecing together various sources of information, I finally ended up with a configuration that met the goals stated above.

News Roundup

NomadBSD 1.3.1

NomadBSD 1.3.1 has recently been made available. NomadBSD is a lightweight and portable FreeBSD distribution, designed to run on live on a USB flash drive, allowing you to plug, test, and play on different hardware. They have also started a forum as of yesterday, where you can ask questions and mingle with the NomadBSD community. Notable changes in 1.3.1 are base system upgraded to FreeBSD 12.1-p2. automatic network interface setup improved, image size increased to over 4GB, Thunderbird, Zeroconf, and some more listed below.

GhostBSD 20.02

Eric Turgeon, main developer of GhostBSD, has announced version 20.02 of the FreeBSD based operating system. Notable changes are ZFS partition into the custom partition editor installer, allowing you to install alongside with Windows, Linux, or macOS. Other changes are force upgrade all packages on system upgrade, improved update station, and powerd by default for laptop battery performance.

New FuryBSD XFCE and KDE images

This new release is now based on FreeBSD 12.1 with the latest FreeBSD quarterly packages. This brings XFCE up to 4.14, and KDE up to 5.17. In addition to updates this new ISO mostly addresses community bugs, community enhancement requests, and community pull requests. Due to the overwhelming amount of reports with GitHub hosting all new releases are now being pushed to SourceForge only for the time being. Previous releases will still be kept for archive purposes.

pf-badhost 0.3 Released

pf-badhost is a simple, easy to use badhost blocker that uses the power of the pf firewall to block many of the internet's biggest irritants. Annoyances such as SSH and SMTP bruteforcers are largely eliminated. Shodan scans and bots looking for webservers to abuse are stopped dead in their tracks. When used to filter outbound traffic, pf-badhost blocks many seedy, spooky malware containing and/or compromised webhosts.

Beastie Bits

Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

Mar 26 2020

39mins

Play

iTunes Ratings

65 Ratings
Average Ratings
59
4
1
1
0

Great for keeping up wth BSDs and Unix

By brysonholland - Mar 19 2019
Read more
They don&#39;t just cover the BSDs, but the wider Unix world as well. Great topics, technical analysis, and interviews. One of the best Unix/BSD podcasts currently out there.

Best beginner-accessible BSD resource anywhere!

By ProCoreyJ - Dec 12 2016
Read more
This is the best source ofBSD info available to beginners but also doesn&#39;t hesitate to dive into the interesting details holding it all together. I have found this podcast to be an essential resource for getting and staying current with tools, techniques, and community to make BSD work for me. Thanks for putting this on! Additionally, I think this podcast is probably the best marketing arm for the community. Keep it available!