Since I had already taken the day off work to get some dental work done in the morning, I figured that I might as well use the rest of the day to check out some history stuff that had been on my list for a while. It was time to head north, to Phoenixville, PA.
This cool town was once the home of the Phoenix Iron Works, a large industrial operation that made a number of iron products. The most interesting ones for my purposes were the 3-inch Ordnance Rifle that was used by the Union army during the Civil War. A large majority of the cannons of this type were made right here in Pennsylvania along the Schuylkill River.
After my recent visit to the West Point Foundry in New York, I felt that I had to see this other artillery factory that was closer to home. In retrospect, I’m really glad that I decided not to wait any longer – as you’ll see in a little bit.
There are a few remnants of the old iron works still around. The most notable is the Foundry Building that was built in 1882. This structure has been restored and now serves as an event venue (with a small museum as well). A few blocks over, the Superintendent’s Office Building still exists as a restaurant. It was vacant during my visit.
Opposite the Foundry Building, on the path of the Schuylkill River Trail, was a “demo” bridge built out of Phoenix Columns – another of the company’s products for creating pre-fab bridges – that dates to 1871. This bridge carries the trail across French Creek.
Sadly, all of these structures were built in the post-Civil War years. The site where the artillery was made was on the north side of French Creek, along the trail. I was able to walk along the path, and even went off on some side trails, but I couldn’t find any ruins that looked to be from the right era. There was a lot of poured concrete, which I think would have been installed later. The iron works eventually made the transition to steel, and continued to operate into the 1970s.
Today, the site where the cannons would have been forged is a new neighborhood called “Steelpointe“. Nothing of these houses existed when I visited in 2019, so I’m glad that I was able to see the site in as “pure” a form as possible. I wonder how many of the people who live here realize how important this stretch of land along the creek was to the Union war effort. I’m constantly reminded of how much history is directly under our feet, if we only have the will to explore it.
Longtime readers will know that I’m something of an artillery nerd, so while I’m “in the neighborhood” I can’t pass up the chance to see where the West Point Foundry was located. This 19th century industrial site manufactured a number of metal goods, but most notably the Parrott Rifle, invented by, and named for, West Point Foundry Superintendent Robert Parker Parrott.
The old foundry grounds have been made into a public park – part historic site and part nature preserve. There is also a pretty cool multimedia tour that can be accessed on-site from a smartphone. I took advantage of that tour during my visit.
The first thing to visit here is a reconstruction of the old artillery testing rig down near the Hudson River. New artillery pieces were tested by firing projectiles across the river toward Storm King Mountain at the very northern end of the United States Military Academy grounds.
I only saw a few other people while I was there – it is quite a serene setting, so the park works well even if you’re only looking for a quiet time in the woods. The waterway that powered the factory is still here – the aptly-named Foundry Brook – and it provides just enough white noise as you stroll along the path of the old railway bed that ran among the various buildings here.
There are several ruins remaining from the days of the foundry. The most notable and prominent is the old 1865 Office Building that was constructed during the height of the site’s productivity. While there are some walls still standing, most of what is left here are building foundations. Even with the multimedia tour and the few wayside markers along the way, you need to use a fair bit of imagination to envision what the place must have been like when it was a major producer of iron goods. Mother Nature has done a good job of reclaiming the land.
The imagination is assisted a bit by the inclusion of a reproduction of part of the old Boring Mill wheel – mainly so the visitor can get a sense of the scale of the thing. This is the part of the factory that made artillery production possible – it carved out the barrel of the guns and allowed for rifling to be done.
I really enjoyed my time at the preserve. There are so many aspects to Civil War history that can be explored – it’s so much more than battlefields.
I’ve spent some time lately redesigning my home network thanks to some new (to me) equipment that I found on eBay. This post is a breakdown of what I’ve done so far (and maybe a bit about my plans moving ahead).
For a few years, I’ve had a really nice router that allowed me to create separate subnets for different purposes. Up to this point, I’d only had small, “dumb” switches, so for each new subnet that I wanted to create, I needed a completely separate router interface and switch. It got out of hand really quickly, becoming a rat’s nest of different consumer-grade equipment.
A few weeks ago, I found a used NetGear “smart” managed switch. In addition to 24 gigabit Ethernet ports, it also has 4 SFP ports for later expansion. All the ports support at least PoE (802.3af) with 8 of the ports supporting the more powerful PoE+ (802.3at) standard. Of course, it allows for VLANs, and link aggregation, too. This one piece of equipment has enabled everything else that I’ve been able to upgrade. It’s a really powerful addition to my home LAN.
I’m not going to lie – this took a little while to get set up. It wasn’t immediately obvious that the aggregate links I was setting up on the switch for my uplink to the router, and for one of my servers, weren’t using LACP by default. This manifested as unreliable links – not broken ones. Most traffic would get through, so it was hard to pin down exactly what was broken when I’ve also just moved EVERYTHING else at THE SAME TIME. I made this move as a total heart transplant as far as the network was concerned, because I felt a lot of pressure to get the Internet back up quickly for the other members of my household. In the end, trying to rush caused me to miss things in the troubleshooting process. There were several cycles of reverting all the changes and starting from scratch. I also could have done a better job of finding documentation and doing a written plan of action first.
In the end, everything worked out, and we have a MUCH more capable network now.
Prior to having the ability to do separate VLANs on the same switch, I had to run totally separate hardware stacks for each subnet I wanted to provide. The same was true for Wi-Fi access: there were multiple access points, each tied to its own subnet and with its own distinct SSID.
Why not put in an access point solution that was also VLAN-aware, and since I now have a PoE-capable switch as well, take advantage of that at the same time? This would also let us get rid of the $15/month we were paying to lease our main Wi-Fi router from our ISP. That was going to be the next phase of the plan. I decided on a Ubiquiti UniFi U6-Lite Wireless Access Point, and was able to find a refurbished one for sale to save a few bucks.
I already had some experience with Docker, so I decided that I would run the UniFi Network Controller software myself within a container. I know – it’s sort of overkill for a single access point setup, but that’s the easiest way to get the VLANs going the way that I wanted. It was super easy to deploy the container, but I had a little trouble getting the access point to adopt at first. It turns out that my VLAN configuration wasn’t quite right on the switch: I really needed a separate management VLAN for the access point and controller to live on so that all the other VLAN tags would work correctly – after a little bit of experimentation, I found a config that worked.
That’s been a theme of this project so far: if I was more experienced with VLANs, or with LAG configuration with the switch, then this whole thing would have been almost plug-and-play. But isn’t that why we have home labs? So we can play with these technologies and gain experience?
Returning to the task at hand – the UniFi Controller software is really wonderful. There is SO MUCH data about your network and how it is running, and it’s beautifully-presented. My kids really enjoy seeing the animated visual representation of traffic flow that can be turned on within the topology section of the dashboard. Very cool stuff!
In addition to our regular “Home LAN” Wi-Fi, we now have separate SSIDs (and VLANs, each with their own appropriate restrictions) for my kids to use, for IoT devices in the house, and even for guests (with a super-slick captive portal built in!) I’m very impressed with the UniFi system.
Servers and Services
So what are we going to do with all these new network capabilities? Well in my last post about this general topic, I talked about a lot of the things that my boys and I had been playing with in our home lab – we’re still doing those things, but at a larger scale and with better organization now.
For starters, I’ve really come to love Proxmox. Since I last wrote here, we’ve upgraded our virtualization server from an old desktop to a used 1U rackmount server that the boys’ grandfather found for us on a local marketplace site. The new machine is a Dell PowerEdge R610 with 2 Xeon X5675 CPUs (for 24 total threads), 80GB of RAM, and 2.4TB of storage. It’s a monster of a machine, so I don’t keep it on all the time – I only use it when I want to play with a really complicated system or set of systems together. I’m able to keep it off because it has an iDRAC integrated lights-out management system that allows me to send a network signal that turns the machine on remotely.
For a few years, I’d been running most of the network services that need to be on all the time on small, inexpensive single-board computers. These have been really great because they don’t use a lot of power or make any noise, and have been reliable enough and fast enough for our purposes. As I’ve gotten more used to having the power of Proxmox around, I’ve started to see the limitations of my old way, though. And similarly to the issue of each subnet needing its own networking, each service that I wanted to limit to a particular subnet needed its own server for that subnet in order to remain truly isolated.
Now that I have proper VLANs, why not set up a lower-power server and run a single trunk connection? Whenever I need to add a new VLAN, it’s as simple as adding a new tag on that server’s interface and the switch port. Proxmox makes that process really easy.
So that’s what I’ve done. I’m replacing 3 separate single-board computers with a single laptop we were no longer using. I put a little more RAM in it, installed Proxmox, and set up a few VMs for hosting internal and external services. All the websites I host for our family are now containerized (including this one). I have a single VM backup to run (which takes just a couple minutes) and all that data is safe. This setup will also make the eventual migration of the sites to new hardware a breeze.
For containerization, I’m still using Docker. I feel like I was a little late to that party, and I had some trouble wrapping my head around it at first, but I see all the benefits now. I know that I can run LXC containers within Proxmox, but I haven’t messed with that too much. I really like the interface that Portainer supplies, and I feel like I have a setup that works really well for our purposes. I might experiment a little in the future (it IS a home lab, after all) but for now, Docker is our “production” setup.
For the Future
Things are working really well for now. I’m so pleased with how everything has come together since ripping out all my old networking and rebuilding around a commercial-grade switch.
I think the next thing I’d like to do is replace my current network storage setup with a proper NAS. I don’t think I’d want to use the UI that comes with the types of systems that you get from something like a Synology or QNAP – I’d probably just use them as a raw pool of storage, and run something like NextCloud for the “friendly” interface to all that storage. I just need a big hard drive that I can talk to over a couple of TCP ports. 🙂
Next, I’d like to improve the reliability of my hosted services by setting up a modest Proxmox cluster. There are a lot of ultra small form factor, so-called “1-liter” desktops on the secondary market – largely as they come off of corporate leases, it seems. My own experience (and what I’ve seen from other home lab enthusiasts) seems to suggest that with enough RAM, they’d work out really well as Proxmox nodes. I’m not sure if I’d go so far as to set it up as a high-availability cluster, but being able to shuffle VMs back and forth as I move around the underlying hardware, or do updates, would be of great benefit.
I’m excited about the possibilities that are available for the boys and I to explore. I’m sure there will be more posts coming as we build more things!
The last few years have been pretty crazy. In addition to the COVID-19 pandemic that the world has been dealing with, I’ve had a lot of things going on in my personal life that have kept me quite busy. While the main focus of my blog here is history, from time-to-time I like to give an update on the things I’ve been exploring in the technology world – mainly because it’s one of those things that my boys and I love doing together. I continue to use the same kinds of open source software that sparked my love of computers in the first place, and that ecosystem has only gotten more robust in the last 20 years. The things my sons and I have set up now have given us a great platform to try out the possibilities and expand our understanding and creativity. It’s been a great way for us to bond.
The boys have been learning about Linux and all the related software by using Raspberry Pi single-board computers. We have quite a collection – and I’m glad because it’s quite hard to find them these days because of all the semiconductor shortages. We’ve had a few Pi 3Bs, a Pi 3B+, a Pi 3A+, and lots of Pi Zeros. Some family members got together to buy the boys a Pi 400 for Christmas the year before last, but it suffered a horrible death due to being transported around too much.
Sometimes, we experiment with microcontrollers, like the BBC micro:bit, Adafruit Circuit Playground, or even Arduinos, and it’s as much a learning experience for me as it is for the kids. If we do anything particularly notable there in the future, I’ll probably make a post about it.
The boys each have their own laptops – courtesy of their maternal grandfather, who has a knack for collecting used computers and reselling them online. Every time we visit, the boys want to “go shopping” in Baba’s basement warehouse for new-to-them pieces of tech gear.
Isaac’s laptop is an Asus X54C with 4GB of RAM and a 500GB HDD. Like his brother, he can boot into Windows 10, but prefers to use KDE neon. Isaac likes to code silly things with Scratch and Microsoft MakeCode – sometimes he loads his creations onto one of our microcontrollers.
I don’t know that I ever really announced it here, but I’m back to self-hosting this website after using a provider for a number of years. The current iteration runs on a Pine A64+ – a really awesome little single-board computer with a quad-core, 64-bit ARM chip and 2GB of RAM – running the Ubuntu Server variant of ArmbianLinux. For resource efficiency reasons, I moved from Apache to nginx for the webserver a few years ago. The site is still built with WordPress. I also host a few other things on this box, including my kids’ websites.
By accident just walking around my local Microcenter one day, I discovered the Ubiquiti EdgeRouter X and couldn’t believe how powerful it looked for the price. I just had to pick one up, and it is now my main router. This has enabled me to run multiple different LANs and segment network traffic in my home to keep IoT devices away from the rest of my computers. While networks seem to intimidate a lot of people, I’ve found the EdgeRouter to be pretty simple and even fun to set up and use. Ubiquiti’s products are great for anyone who wants to take their home network to the next level.
Along the lines of networking, I wanted to have a VPN set up so that I could have secure access back to my home network if I needed to modify anything on the go, or just to encrypt my traffic for security reasons if I had to use public wi-fi. I set up OpenVPN on a dedicated server for this purpose and have absolutely no complaints about the way it performs.
I’m certainly no expert on containers – I’ve only begun playing with Docker recently on a Raspberry Pi 4 with 8GB of RAM that I’m running as a server – but I can see why it’s become a popular technology. Managing the containers through a web interface – I use Portainer – has made it much easier to understand what is going on. The boys and I have played with a few things in Docker:
Guacamole – An open source VNC / RDP / SSH gateway that can run on basically anything with a modern web browser. Very cool technology, but it was a bit of a resource hog on the Pi.
PiHole – I’m a little bit paranoid about “smart” TVs, so in addition to running a totally separate network segment for my IoT devices, I also funnel their network requests through this DNS filter to cut down on the amount of spying that these devices are capable of (or even like to do). This also has the benefit of cutting down on the amount of ads we see on websites, and I have it configured to protect us from other harmful stuff out on the Internet.
Habitica – A tool for making real life self-improvement into something like a video game. The “open source” version forces you to clone their entire website – including all their payment processing code – and really seems to be intended for people who want to help them fix their bugs. That said, this was a little too clunky to be useful as a self-hosted thing in our case.
Grocy – We set this up as a home inventory system for groceries. Still experimental for us at this point. I love the idea of being at the grocery store and knowing how many cans of soup we have at home, but keeping the inventory up-to-date is where we have a problem.
I haven’t quite gotten to the point where I feel comfortable running the things I really care about within containers, but maybe that will come some day.
The REAL fun has been in exploring Proxmox – an open source hypervisor server based on Debian. Admittedly, virtual machines are definitely heavier on resources than containers, but they provide a lot more flexibility in my mind. Through a web interface, we can now quickly spin up virtual servers to play with different operating systems, software, and even network configurations – and it’s a lot easier to tear them down once we’ve finished.
I’m running Proxmox on a second-hand Dell XPS 8300 Desktop with 16GB of RAM and that has been plenty for our purposes. So far, I’ve kept a dedicated FreeBSD VM running on it, as well as a VM for playing with Kali Linux. The boys have also used it for test driving different Linux distros before they each chose one for their own laptops.
I’ve decided to change things a little bit around here. I’m switching the web address for this site over to a domain I’ve had for a few years and just haven’t done anything with. As of today, the new official URL for this blog is:
Since I was a kid, I’ve loved road-side historical markers. I always wanted to stop and read them, and sometimes (when we weren’t in too much of a hurry) I got the chance to. There’s something really great about seeing tangible reminders of history out in the world where you’re living.
It turns out that I’m not the only person who feels this way. Several years ago, I discovered the Historical Marker Database – a hobby project of a history-loving IT guy like me – that seeks to catalogue every historical marker in the world. I make heavy use of the website when I’m researching, and it also makes for a fun way to go down a historical rabbit hole that I might not explore otherwise. You should definitely go check it out.
As you might imagine, finding all these markers is a huge undertaking – certainly more than one hobbyist can handle. A volunteer board of editors has sprung up over the years, and thousands of people have contributed photos and descriptions of markers to the cause.
Anyone who runs a WordPress site knows about comment SPAM.
Basically, there are computerized robots roaming the Internet, looking for WordPress (and other blogs) to leave comments on. These comments are usually in the form of links to porn websites, or to sites selling fake Louis Vuitton bags out of Yugoslavia or something. The idea is to get links to their site on as many other sites as possible, because that’s how they increase their ranking on Google.
Anyway, I get a lot of this stuff. So much in fact that I can’t leave comments turned “on”, they have to be left in a “moderated” mode, where every comment needs to be approved before it is visible. It’s a mess.
It’s not all dark clouds, though. The entertaining thing is that there aren’t just links in these comments. They try to put something in there that sounds like a legitimate comment, but is always just a little off. Here are a few examples that I found particularly funny:
“A real uncle isn’t friends, although friends are usually each uncle.”
“Fancy can be the effective requirement for your personal living therefore the expansion of truley (sic) what most of us take pleasure in.”
“When you really need any accountancy of one’s importance, rely pals.”
“Around the world you most likely are someone, yet to person you most likely are everything about.”
Actually, there might be some good advice in there….
This is the 5th “reboot” of my personal website in the last 12 years. I just can’t seem to find a format that I like.
That first site was put up in December of 2000 – the original SkilmNET – a combination of my high school nickname, “skilm”, and “NET” that was so popular back in those days (cNET, PSI-NET, etc.).
I was utterly fascinated with computers and the Internet. I had been introduced to the Internet at school in 1997-98, and when I finally convinced my parents to get a home Internet connection, I was mesmerized. I HAD to know how this magical thing worked. I spent hours exploring the possibilities – websites, email, instant messaging, video, the whole lot. I was so enthralled that I decided to study Computer Information Systems when I started at UMBC in the fall of 1999.
What I didn’t expect was that I would be SO FAR behind my peers in the program. I had been a musical theatre nerd in high school – a performer. I wasn’t good at math. I had never even physically opened up a computer before. Now, I was in programming classes desperately trying to learn C with hardcore nerds who had been doing all of this since they were pre-teens.
It was a disaster.
The silver lining is that as part of that C programming class, I was required to log into a UNIX server at UMBC: gl.umbc.edu (which is sadly shutdown now – I’ll never forget that box). You see, normal human beings don’t have computers with C compilers on them – let alone UNIX computers – so the university provided everyone with a shell account on gl.umbc.edu that you could access remotely via telnet. You would log in, do your work on their computer, and turn in your finished program.
YOU would do this, but so would EVERYONE ELSE. The system got SLOW. Especially when a big project was due. I was already aggravated enough that I had to learn this cryptic C language – I didn’t want to suffer with a slow computer while I did it. I knew that the university was using Linux and gcc to teach us. I knew that those software packages were freely-available. I knew that I could get hold of an old computer to use for this project. I thought, “How hard can it be?”
At the time, our family had just one computer in the house. I was thinking about adding another. This meant that I needed to learn a little bit about networking so that I could split the Internet connection. About 4 hours of head-scratching later, I learned that a hub was not enough – I would need a thing called a “router”.
In the meantime, I was able to get RedHat Linux (back before it was called Fedora) and the gcc compiler installed on an old, no-name Pentium 133MHz PC. It certainly wasn’t setting any speed records, but I was the ONLY user. Compared to my gl.umbc.edu experience, it was a rocketship.
I was so happy with it, I decided to let a few of my friends in the class in on the action. I figured out how to set up user accounts and allow telnet access to my buddies. When my friend Dave logged in from the other side of town, and I saw his username show up in my who command – it was amazing. My best friend since elementary school was using MY computer. From MILES away.
Over the next several years, I installed every service that I could possibly find on that box. I set up a webserver – complete with individual sites for each of my users. We had FTP. Shell accounts (over telnet or SSH). I learned enough to get an e-mail server going – it REALLY started working right once I learned about this crazy DNS thing. I got sick of being limited by the little Linksys home router that I bought, so I learned how to set up DHCP, routing and ipchains (later iptables) on another Linux computer. While I hated my database class in college, I LOVED the thrill of getting MySQL and PHP running on my own little piece of the Internet. I kept changelogs. I sent out e-mails to a Majordomo list of my half-dozen users warning about planned downtime for upgrades. I became the go-to guy for technical needs.
I advanced in my part-time job at the Baltimore County Public Library. I wasn’t shelving books anymore – I was doing tech support for the dial-up ISP service the library ran and learning everything I could about our Solaris UNIX backend systems. I became addicted to getting more and more bizarre computers for my network. I branched out from x86 with my first SPARC box – a SPARCstation 20 (we ran SPARC Solaris at the library). After I left the library to work for AppleCare, I bought a PPC G4 iMac. That led to an old NeXT Workstation. Then another SPARC box (this time a Sun Ultra 60).
Somewhere along the line, I upgraded my original Linux box to a Pentium II 400MHz. I kept finding new things to learn and set up: webmail, MRTG, applications to track my system logs, more robust remote management tools, more users. I even set up WordPress (which this site is running on) once or twice.
Eventually, once I took the job as the solitary computer guy at a small company, my hobby became too much like my job. I couldn’t keep up with it at home anymore. The friends who were my users drifted away, as often happens in life. I met the girl who would become my wife (as interesting as technology was, she was even more interesting). At the same time, the world changed. I didn’t NEED to host my own stuff to get what I wanted out of technology. The rise of services like LiveJournal, Flickr, Facebook, .Mac (later MobileMe, even later iCloud), YouTube, Dropbox, and Gmail meant that I didn’t have to use my own systems so that I could get cheap “unlimited” storage on the Internet. I got a phone that lets me use all these services, wherever I am, with ubiquitous wireless networking. The only reason to do it myself was for pride, and I just didn’t care enough about it anymore.
I kept the domain name and email addresses (which I was still using) alive by getting a hosting service. And it has just sat there – nearly idle – for years.