PETSCII typer: type your favorite C64 characters

▼ A few months ago, I ordered the Drop + Matt3O MT3 retro keycap set:

These keycaps have the cool/retro "PETSCII" characters printed on the front. But how do you actually type those PETSCII characters?

The first part of the solution is the C64 Pro Mono font by style64.org. This font replicates all the characters from a Commodore 64.

My web-based PETSCII typer tool connects everything together by producing the correct characters as you type as if your keyboard is a C64 keyboard, or using the layout of the MT3 keycap set. You can even load PETSCII text files or C64 BASIC programs which will show up more or less correctly. Or convert to current fonts to the degree possible.

When using the C64 Pro Mono font, if you copy some PETSCII text and then paste that text, it will use the C64 Pro Mono font if that font is installed on your system. This way, the PETSCII looks as intended in a word processing document.

There are still a few minor issues to be figured out, but I wanted to get this out sooner rather than later. Send me an email or a Twitter message if you have any comments or questions.

Permalink - posted 2022-03-23

Making multilingual typing easier with just one new key

▼ Although in the 1980s and 1990s computer were capable of displaying text in many languages, they were limited to one set of characters at a time. So either Western Europe, Eastern Europe, Greek or Cyrillic, but not several of those at the same time. Unicode solved that, and modern computers can (in principle) display all characters found in all languages.

However, when it comes to typing those characters, we're still in 1990, with different keyboard layouts for different languages. Now obviously it would't be workable to make a keyboard that lets you type all Unicode characters. But it should be doable to come up with a system that provides access to all characters and diacritics that are used in latin script languages.

All it takes is one new key. I call it the globe key: 🌐

We'll have to repurpose one existing key to make room for the globe key. On US keyboards, I think the ` key (in the top left corner, left of 1) is a good candidate, as the ` character doesn't occur in normal text, and even in programming it's not used very often. (But don't worry, you can still get ` by typing 🌐 twice.)

The idea is that the globe key gives you access to a new keyboard "level", similar to how the shift or alt / AltGr keys provide access to additional characters, or different forms (uppercase) of the regular characters. When you type 🌐, the next key that you press will give you the following:

Just like the globe key itself, the keys for diacritics (accents) are "dead" keys. So if you type one of those, nothing happens just yet, until you type the letter you want the accent to appear on (or under). So if you want to type "café", that would go as follows:

c a f 🌐 ◌́ e

Now obviously, if you regularly type French, where the acute accent is very common, you wouldn't want to type it like this every time. You'd keep your existing AZERTY layout (or other keyboard that you're using right now) which has a dedicated é key. But what if you need to type an Eastern European name, such as Michał, Enikő or Věra? The globe key comes to the rescue:

🌐 ◌̷ l = ł
🌐 ◌̋ o = ő
🌐 ◌̌ e = ě

How this is different from other approaches

When computers started to become common, various countries were already using different keyboard layouts on typewriters. And that makes sense, as in addition to regular letters and digits, there's only a dozen keys available for punctuation and accented or special characters. So each of those really needs to earn its keep to deserve to get a key of its own, and languages vary wildly in which characters and accents are common.

Since the 1990s, ISO/IEC 9995 has been working on standards in this area. But you're forgiven for not noticing. It looks like so far, only the multilingual Canadian CSA layout based on ISO/IEC 9995 has gained any traction. EurKEY is a more informal effort, but seems to have stalled.

And of course very many people have made their own keyboard layouts that work for them. Others switch between different layouts to be able to type characters from different languages. And of course simply ignoring the issue and letting a spell checker take up the slack is also a widely used strategy.

Unless I'm mistaken, the US keyboard layout is the only one that doesn't use alt (AltGr / right alt on Windows) for typing certain characters. Both ISO/IEC 9995-3 and EurKEY specify a specific alt / shift+alt "level" for typing additional characters. However, the issue with that is that users need to learn the new positions of these characters if they adopt those layouts.

And although spending a bit of time learning something new that is more efficient is usually time well spent, people simply don't like change. And in the real world, often the new and the old exist side-by-side, so that means having to work with different keyboard layouts on different computers. Not great.

Now, I'm all for replacing the more or less random alt / shift+alt levels that are part of existing keyboard layouts with the new one shown above. If you want to do that, make your alt keys or AltGR key the globe key. In that case, you'll have to press the alt / AltGr = globe key along with the next key.

But if you're used to a layout that lets you type certain special characters or accents easily with alt / AltGr, you may not want to be forced to learn a new way to do that. So in those cases, it makes more sense to just sacrifice one existing key to get the globe key so you can type characters not supported by your existing layout, but otherwise keep typing the way you're used to. If you put the globe key somewhere else than under alt / AltGr, you need to press and release it before you press the next key.

Why this layout makes sense

The globe key layout is designed to work well for characters and diacritics that you don't type very often. So the ease of typing takes second place relative to the ease of remembering. As such, the layout is as straightforward as possible.

People used to the US and similar layouts will find ` ~ ! @ # $ % ^ & * ( ) - _ = + [ { ] } | ; : ' " < > / ? where they expect them to be after pressing 🌐. The cedilla, grave, hook and ring accents and (inverted) question mark are found under the keys matching their first letter. Acute, diaeresis/umlaut, tilde and (arguably) vertical stroke are found under the letters that most frequently bear that accent: é, ü, ñ, ł. Circumflex, comma below, dot below are under the keys that look the same. Accents below are all on the bottom row. Similar looking accents are next to each other or above/below each other.

The above applies when using a QWERTY layout. So æ is between caps lock and S, as that is obvious on a QWERTY layout where that key is A. On an AZERTY layout, that same key is Q, but after pressing 🌐, that same key still produces æ. This makes learning the globe layout a bit harder to learn for AZERTY typists, but the advantage is that once learned, it applies the same everywhere.

Whenever possible, the 🌐 key should be blue to make it easy to find.

Permalink - posted 2022-03-06

Longevity of recordable blu-ray discs (BD-R / BD-RE)

▼ So now I got that new optical drive and can start burning CDs, DVDs, BDs (regular 25 and 50 GB blu-ray discs) and BDXLs (newer 100 and 128 GB BDs). But how long will those last?

For each of these types of optical disc, there are three types: mass produced, recordable and rewritable. The mass produced ones are made by stamping physical pits and lands into polycarbonate, cover this with a reflective aluminium layer, and then a lacquer (CD) or polycarbonate (DVD/BD) top layer. DVDs and BDs can have two data layers, BDXLs have three and sometimes four, with semi-transparent reflective layers.

Recordable CDs and DVDs (CD-R, DVD-R and DVD+R) have a layer of photosensitive dye below the reflective layer. The CD/DVD writer uses its laser at higher power to "burn" into the dye layer. With the result that even early CD drives can play CD-Rs without issue. DVD+R improves somewhat on the burn process used by DVD-R.

I'll cover BD-Rs/BDXL-Rs below.

Rewritable or recordable/erasable CDs (CD-RW), DVDs (DVD±RW), BDs (BD-RE) and BDXLs (BDXL-RE) all use a phase change material layer to hold the data. Depending on the intensity of the laser, the phase change material may end up in an amorphous or a crystalline state, which each having a different optical properties.

Longevity: the reflective layer

Obviously all types are sensitive to scratches from poor handling, but the advantage of optical discs over vinyl records and tape is that playing/reading them doesn't involve any contact with the part of the disc that holds the data. So mass produced discs can be played an unlimited number of times without any degradation. For the recordable types, there is a possibility that over time, the reading laser will affect the data layer. But unless you're playing one of these on repeat for months on end, that shouldn't be a problem in practice.

It's widely recommended to store discs in jewel (CD) cases or similar cases where the disc is only held by the hole in the middle, so there is nothing touching it or exerting pressure. And store them vertically.

For mass produced discs, the main issue is that sooner or later, oxygen will reach the aluminium reflective layer, which will then start to corrode and eventually render the disc unreadable. This disc shows lots of small holes in the reflective layer, but so far it plays just fine.

Recordable/rewritable discs often use silver or a silver alloy for their reflective layers. Silver won't corrode immediately just from being exposed to oxygen, but it will in the presence of pollutants like sulfur oxides.

According to Verbatim, a maker of such discs, it's possible to tell the difference between an aluminium and silver blank discs by holding them up to the light and seeing if there's any light coming through the disc. If there is, the reflective layer is silver. If there isn't, it's aluminium.

Of course this doesn't work if there's an opaque layer on top of the disc, for instance, if it's a printable disc. These are pretty nice, by the way:

Longevity: the data layer(s)

The photosensitive organic dye layer in CD-Rs and DVD-Rs is also somewhat of a liability: it will degrade over time. CD-Rs come in various colors that may or may not indicate the type of dye used, with better ones being developed later on.

DVD±Rs always look purple but of course came along when better dyes had been developed, anyway. In practice, I found that out of about a dozen CD-Rs that I burned between 2000 and 2004, one just wouldn't be recognized at all and two others had problems reading a few of their files. For DVD-Rs and DVD+Rs that I burned between about 2003 and 2010, I had issues with about 5% of the discs I tried to read recently.

Rewritables already have the issue that the difference between the two states of the data layer material are much less obvious, but it must also be able to change in two directions, making the addition of room for error much harder. In practice I've had relatively poor results with CD-RWs and DVD-RWs. Sometimes they won't even verify after a burn, and ones from the 2000 - 2010 timeframe had a failure rate of more than 75% when I tried to read them recently.

No problems so far with a box of fresh 25 GB BD-REs, though.

BD-R HTL vs LTH

Which brings us to recordable BDs/BDXLs. Unlike their predecessors, those originally didn't use an organic dye layer to record data, but a composite material, which is more stable. These discs are referred to as high to low (HTL), which gets at the reflexivity of the disc: a blank disc has high reflectivity, which becomes lower when burning.

However, a few years in the disc manufacturers brought the organic dye approach to BD-R, which has the property that the blank disc has low reflectivity, and then gets more reflective after burning, so "low to high" (LTH). This way, the manufacturers could reuse their existing CD-R/DVD+R production lines with only small modifications. HTL discs, on the other hand, require a more complex sputtering production method.

The way I see it, the only reason to still use optical for storage is because of its longevity. The good part is that you can replace a drive and then still read all your existing discs. With hard disk drives and flash, once one part fails (mechanics? electronics? the data?) then you're pretty much left without any useful course of action. In that light, the adoption of LTH BD-R is a step back.

A Reddit contributor felt much the same, but also concluded that LTH BD-Rs had completely replaced HTL BD-Rs in 2018. (Worth reading just for the first comment: "This is one of those posts that shows that it's entirely possible to be 100% correct, and also 100% bat-shit crazy.")

Fortunately, things don't seem to be quite this bad. Yes, it's annoying and perplexing that disc manufacturers don't label their products with important information like whether the discs are HTL or LTH. But, it does seem HTL BD-Rs are still being made, as I ordered a spindle of 10 25 GB Verbatim 6x speed inkjet-printable BD-Rs before I did any research... and it looks like those are HTL. Using Apple's System Profiler, we can see the following under Disc Burning:

We can then look up the ID over at the Disc Manufacturer ID & Media Type ID Licensee List and find out that the CMCMAG BA5 disc type is actually HTL.

Also, if you're careful (it's easy to get confused), you can tell by looking at a burned disc whether it's HTL or LTH. The two discs at the top are a blank DVD+R and a used DVD-R. The ones at the bottom a blank BD-R and a used BD-R.

If you look carefully, you can see that the unused part along the edge on the top right DVD-R is slightly darker than the burned part. For the BD-R at the bottom, it's the other way around, which is easiest to see around the middle. The first ring is very dark, but that one isn't writable. The next ring is writable material, but outside the writable area. This doesn't show up all that well on photos, and it's also easy to misinterpret. But there is a visible difference.

M-DISC

M-DISCs are supposed to last for 1000 years. The accelerated aging tests aren't proving them wrong, but of course also not proving them right. (Wait another 990 years for that.) But at least M-DISCs are making a claim about longevity, which is more than the competition does.

What to do?

If you have data that you want to stay around for a long time, without having to copy it from one HDD or SSD to another every few years, optical is probably the best choice. Still, just writing your data to disc once and then assume it will remain readable for decades is too optimistic. So I'd say:

• If possible, check the error levels for every disc you burn. Unfortunately, this information is generally not available.
• Store everything at least twice, preferably three times. Apply optical disc storage best practices.
• Use different types of discs and/or storage for the different copies.
• Also try to keep the different copies in different locations, both to accommodate for large scale disasters such as fire and floods, but also more mundane stuff such as less than ideal temperature and relative humidity.
• Check your discs once every decade or so, and make fresh copies when in doubt.
• Single layer (25 GB BD) is probably a safe bet, with less to go wrong compared to multiple layers.
• BD-R M-DISC is probably best, with BD-R HTL in second place. Whether BD-RE is better than BD-R LTH is unclear. So when in doubt, use both.

Permalink - posted 2022-01-09

Optical storage and media in the '20s

▼ Last year I wrote posts about listening to the radio, watching TV and having a landline phone in the 2020s (in Dutch). Today, I want to talk about the place left for optical storage in the 2020s.

Do I actually want or need those shiny optical discs in my life?

The short answer: most people probably don't. Software is now always available online, so you don't need an optical drive to install applications or drivers. For music, you can get pretty much everything through streaming services such as Spotify. And if you bought CDs back in the 1990s, you'll certainly have ripped those by now. I think you can watch pretty much any movie on one online service or another. Current TV shows are also generally available on a streaming service.

Transitioning to the disc-free lifestyle

But what if you bought movies and TV shows on DVD in the past, and used to burn some of your data to CDs and/or DVDs? You really want to go over all your data CDs and DVDs and copy those to a harddrive while you still have a working optical drive. That optical drive will die at some point (or become unreliable to the point of being unusable). You can of course then spend less than $/€ 50 for a DVD drive and handle this chore then, but why wait? Those CD-R and DVD±R organic dye layers aren't getting any better as they age.

Ripping movies: meh

I'm not sold on the option to rip all my movies and TV shows on DVD and blu-ray to my NAS or an external HDD. That's just way too much data. For movies, I don't have too much of an issue with simply using my blu-ray player (the one connected to my TV) to watch them. Yes, finding the right disc and then waiting for the drive to do its thing and having to deal with unskippable warnings and annoying menus isn't great, but that's only a minute or so vs a movie that's at least 90 minutes.

Ripping TV shows the hard way

For TV shows, this is quite different. Especially when I want to watch a 22 minute sitcom episode (and boy, do I love those turn-of-the-millennium sitcoms!), having to wait for the player to do its thing and then the warning and menus and me forgetting which episode was next, it's all quite annoying.

So what I used to do before my the Apple USB Superdrive that I got in 2011 kinda-sorta died recently, was rip TV shows from DVD and then use Handbrake to convert them to H.264 format, put in all the iTunes metadata using Subler, and store those on my NAS. However, that is quite a lot of work and eats up quite a bit of storage.

Ripping TV shows the easier way

Then I found an easier way to handle TV shows on DVD (and, to some degree, blu-ray): MakeMKV. MakeMKV is an app that, well, makes MKV files. You can use it to rip DVDs (easy), regular blu-ray (a bit more difficult) and UHD/4K blu-ray (when the stars align). My new blu-ray burner mentioned below didn't give MakeMKV any trouble when ripping DVDs, even region 1 ones on this region 2 drive.

For DVDs with TV shows, it's relatively straightforward to select the right tracks and turn those into .mkv files. However, MakeMKV doesn't do any transcoding. So the video is still MPEG 2, and the audio is still the same (usually) AC3. This means the files will be unplayable in many places, including iTunes and the iPhone/iPad.

However, my LG TV is more than happy to play these files. And, being a TV, it handles the intricacies of framerates and deinterlacing well automatically. fSo I just rip one DVD at a time, put the MKV files on a USB flash drive, and stick that in one of the USB ports of my TV. Yes, the files are three times as large as compared to the handcrafted H.264 versions, but I don't have to think about any settings and just delete the files after watching. Should I want to rewatch the same show a few years from now, I can always rip the same DVDs again.

My new optical drive

Like I said, my Apple USB Superdrive is no longer working properly, and the idea of having a big box full of music CDs and a smaller box full of data CDs and DVDs that I can no longer read bothers me.

So I decided to get a new external USB optical drive. One that can read and write blu-ray discs in addition to CDs and DVDs, of course. I also had a look at UHD blu-ray, which is used for 4K content on blu-ray. However, reading those requires firmware modifications, if possible at all. Dive into the MakeMKV forums to figure that out.

So I ended up ordering a Hitachi-LG BP55EB40 "Slim Portable Blu-ray WRITER" for a little under € 100. Optical drives used to have some issues with the amount of power provided by the USB2 bus in some cases, but that doesn't seem to be a problem anymore. You'd think USB3 would solve that, but no, as the datarates required for 6x speed BD, 4x speed BDXL (100/128 GB blu-ray), 8 x DVD and 24 x CD reading and writing fall well below what USB 2.0 can handle, that's what we get. The drive pretty much handles all the formats:

It will also write M-DISC discs. Those are recordable DVDs or BDs that use a special material for the recordable data layer(s) that should make them last for a thousand years.

Blu-ray for archiving

I'm not sure if my data still needs to be available in the year 3022, and it probably won't be, even on an M-DISC, but given that my failure rate from the DVD-Rs that I burned between about 2003 and 2013 is quite low (maybe 1 in 25 is no longer readable), an improved type of recordable disc seems like a good bet for things that need to last a few decades.

The advantage of an optical disc as an archival medium is that it only includes the data, not any electronics or logic. It's also fairly small and especially flat, so you can mail them or store them in a folder or safe deposit box and not worry about shocks or drops. A couple of years ago, I wrote about using flash for archiving, purely based on price. However, as flash capacities have gotten larger, especially for USB thumb drives and SD cards, their reliability has dropped. This is not a technology appropriate for long time archival storage.

Also, unlike storing data on a (possibly archival) hard disk drive (HDD), burning a disc is a one-time thing, so it encourages careful consideration of what needs to be archived in what format.

Conclusions

Although I think it's possible to leave the optical past behind, I do think keeping an optical drive around to rip TV shows and burn archival copies of my data is a good idea.

Permalink - posted 2022-01-02