For this round of Miscellaneous Findings, we have things related to the internet, mostly the doings of servers.
This is a roundup of miscellaneous things that I’ve found out about (or have rediscovered). I take notes on findings regularly, and I put the findings that translate well to speech on my podcast, Small Findings. The rest (which are often technical findings), I put here. They’re not always written up for maximum comprehension as a blog post, but if anything is hard to understand, please email me if you need clarification.
ncat is a command that can be used to watch network connections and pass requests to a script to respond!
For example, this would pass http requests to
ncat -e respond.sh -kl 80
respond.sh can then use
read to put stdin into a variable (e.g.
read -r request _) then write the http response to stdout.
It’s minimum viable web server!
#unix #command #network #bash
If you see traffic from Boardman, Oregon in your server logs, keep in mind that there are at least four Amazon data centers at Port of Morrow in Oregon, which is near Boardman.
Web site delivery speed
You can easily find out how fast your web page loads (and how fast it becomes usable) to your own computer by simply loading your web page in your browser. Sadly, most people aren’t loading your web page from your computer. So, you need to find oUt how it loads on other computers somehow.
Testing web page speed
Tools for finding out how fast a web page can be loaded and used throughout the world:
- PageSpeed Insights for speed that doesn’t factor in user location.
- GeoPeeker to get a sense for how things load in other parts of the world.
- It often doesn’t work at all, though.
- There is ShotSherpa, but it doesn’t give you load times.
- GeoScreenshot gets times, but it’s a little harder to see.
- No Africa in there.
- Puppeteer scripts to measure performance
Conventional wisdom says that if you want better global performance, you should just get a CDN.
CDNs are mostly expensive. Netlify is cheap but has other costs.
Comparison of load times: single server vs. CDN
The home page at jimkang.com is served from a single conventional server. I put a copy of it on Netlify so that it could be served via Netlify’s CDN. Then, I compared the conventional and CDN load times from clients around the world, using GeoScreenshot.
This was only run once and without looking hard for possible confounding factors, so it’s an anecdote, not incontrovertible proof. Just take it as directional information worth digging into more deeply.
Netlify CDN version: https://naughty-dijkstra-3087c7.netlify.app/
Normal version: https://jimkang.com
On GeoScreenshot, load times in seconds:
The CDN was faster in Canada, India, and China in this run, but not anywhere else. (I couldn’t get African or South American location in there, unfortunately.)
In this set of load time comparisons, I used Observatory, a large JS- and data-heavy web app.
Netlify version: https://observatory.netlify.app/
Normal version: https://jimkang.com/observatory/
The CDN was faster in India and the US in this run, but not anywhere else.
So, CDNs don’t always make a difference. To be clear, I’m not saying they’re some sort of scam; they usually make a big difference when serving media files. They’re not a silver bullet it all situations, though. Definitely test before shelling out cash.
Reducing your page size, however, is a very reliable way to improve your global load times.
#web #usability #performance
Playwright, a headless browser runner, like Puppeteer, might “just work” on a laptop, as it did for me. But it does have a lot of dependencies, which might not be there, as it was in the case of my VPS instance.
# 2. Install WebKit dependencies RUN apt-get update && apt-get install -y --no-install-recommends \ libwoff1 \ libopus0 \ libwebp6 \ libwebpdemux2 \ libenchant1c2a \ libgudev-1.0-0 \ libsecret-1-0 \ libhyphen0 \ libgdk-pixbuf2.0-0 \ libegl1 \ libnotify4 \ libxslt1.1 \ libevent-2.1-6 \ libgles2 \ libvpx5 # 3. Install gstreamer and plugins to support video playback in WebKit. RUN apt-get update && apt-get install -y --no-install-recommends \ libgstreamer-gl1.0-0 \ libgstreamer-plugins-bad1.0-0 \ gstreamer1.0-plugins-good # 4. Install Chromium dependencies RUN apt-get update && apt-get install -y --no-install-recommends \ libnss3 \ libxss1 \ libasound2 \ fonts-noto-color-emoji # 5. Install Firefox dependencies RUN apt-get update && apt-get install -y --no-install-recommends \ libdbus-glib-1-2 \ libxt6 # 6. Install ffmpeg to bring in audio and video codecs necessary for playing videos in Firefox. RUN apt-get update && apt-get install -y --no-install-recommends \ ffmpeg # 7. Add user so we don't need --no-sandbox in Chromium RUN groupadd -r pwuser && useradd -r -g pwuser -G audio,video pwuser \ && mkdir -p /home/pwuser/Downloads \ && chown -R pwuser:pwuser /home/pwuser # 8. (Optional) Install XVFB if there's a need to run browsers in headful mode RUN apt-get update && apt-get install -y --no-install-recommends \ xvfb
#automation #browser #configuration