Unicode Filename Support in TAR Files in Windows 10

No Unicode Filename Support in ZIP Files in Windows 10

I recently discovered that the built-in ZIP file support in Windows 10 does not support Unicode filenames. For example, I had a file name “みんな一列に.jpg” and tried to add it to a ZIP file, but got the following error:

No Unicode Filename Support in TAR Files in Windows 10

Windows 10 has added built-in support for TAR files through a command line tool. Since I was unable to put files with Unicode filenames into ZIP files in Windows, I tried using this tool to put them into a TAR file. I used the following command to add the “みんな一列に.jpg” file to a new TAR file called test.tar:

D:\Temp>dir /b *.jpg
みんな一列に.jpg

D:\Temp>tar -cvf test.tar *.jpg
a ??????.jpg

Unfortunately, it replaced the Unicode characters with question marks (?). This replacement of characters can also be seen when performing the following command:

D:\Temp>tar -tvf test.tar
-rw-rw-rw-  0 0      0       37044 Nov 10  2004 ??????.jpg

WSL to the Rescue

Windows 10 has the ability to run UNIX command line tools using the Windows Subsystem for Linux. See https://docs.microsoft.com/en-us/windows/wsl/install-win10 for instructions on how to install the Windows System for Linux. Linux also contains a command line tool for creating TAR files. I attempted to create a TAR file containing the “みんな一列に.jpg” file using the following command in an Ubuntu bash shell:

matthew@KIKI2015:/mnt/d/Temp$ ls *.jpg
みんな一列に.jpg
matthew@KIKI2015:/mnt/d/Temp$ tar -cvf test.tar *.jpg
みんな一列に.jpg

I first verified the filename was preserved using the following command:

matthew@KIKI2015:/mnt/d/Temp$ tar -tvf test.tar
-rwxrwxrwx matthew/matthew 37044 2004-11-10 00:54 みんな一列に.jpg

I then extracted the file to a new folder using the following command:

matthew@KIKI2015:/mnt/d/Temp$ cd test
matthew@KIKI2015:/mnt/d/Temp/test$ tar -xvf ../test.tar
みんな一列に.jpg

It even appeared correctly in Windows File Explore:

Back to Windows 10

I was curious to see how the Windows 10 tar program would react to the TAR file created in Linux, so I tried listing the contents of the TAR file using a Windows 10 command prompt window:

D:\Temp>tar -tvf test.tar
-rwxrwxrwx  0 matthew matthew 37044 Nov 10  2004 pü+péôpü¬S+Çsêùpü½.jpg

Unsurprisingly it did not interpret the filename correctly. Interestingly, extract the file resulted in yet a different filename: みんな一列に.jpg.

Hopefully Microsoft will add support for Unicode filenames in ZIP or TAR files in a future update to Windows, but until then, WSL can be used.

How to install Java, SDKMAN!, Groovy, and Grails on the Windows Subsystem for Linux

This article explains how to install various development tools using the new Windows Subsystem for Linux that is available on Windows 10.

Install Windows Subsystem for Linux

In order to use the Windows Subsystem for Linux, it must be installed. The following link explains how to install this functionality in Windows 10:
https://msdn.microsoft.com/en-us/commandline/wsl/install_guide

Update Bash on Ubuntu on Windows 

Once installed, it is a good idea to update to the latest version of the Ubuntu binaries. Issuing the following commands from the Bash shell will accomplish this:

me@COMPUTER:~$ sudo apt-get update 

me@COMPUTER:~$ sudo apt-get upgrade 

Source: https://msdn.microsoft.com/en-us/commandline/wsl/faq#how-do-i-update-bash-on-ubuntu-on-windows

Install zip and unzip

Before you can install the Java JDK or any of the other tools, zip and unzip will need to be installed.

Install unzip

To install unzip:

me@COMPUTER:~$ sudo apt-get install unzip

To verify unzip was installed correctly:

me@COMPUTER:~$ unzip -v

Install zip

To install zip:

me@COMPUTER:~$ sudo apt-get install zip

To verify zip was installed correctly:

me@COMPUTER:~$ zip -v

Install the latest Java JDK

A nice shell script that can be used to install the Java JDK on the Windows Subsystem for Linux can be found on the following Stack Overflow article:
http://stackoverflow.com/questions/36478741/installing-oracle-jdk-on-windows-subsystem-for-linux#41072208

The URL for the latest version of the Java JDK can be found at: http://www.oracle.com/technetwork/java/javase/downloads/index.html

To verify which version of Java was installed:

me@COMPUTER:~$ java -version 

Install SDKMAN! 

SDKMAN! is a tool for managing parallel versions of multiple Software Development Kits on most Unix based systems. Instructions on how to install SDKMAN! can be found on the SDKMAN! website:
https://sdkman.io/install

Once SDKMAN! is installed, it can be used to install other tools, like Groovy or Grails.

Install Groovy

To install Groovy, issue the following command from the Bash shell:

me@COMPUTER:~$ sdk install groovy

Install Grails

To install Grails, issue the following command from the Bash shell:

me@COMPUTER:~$ sdk install grails

Once grails is installed, the steps outlined in the Grails documentation can be followed to create a “Hello World” application:
http://docs.grails.org/latest/guide/gettingStarted.html#creatingAnApplication

Note: I had to create the helloworld directory and navigate to it before I issued the create-app command:

me@COMPUTER:~$ mkdir helloworld
me@COMPUTER:~$ cd helloworld
me@COMPUTER:~/helloworld$ grails create-app helloworld

Once I created my Grails application and the HelloController, I was able to start my application from either the Bash shell or the Windows Command Line using the following commands:

Bash Shell

me@COMPUTER:~/helloworld$ ./gradlew bootRun
:compileJava NO-SOURCE
:compileGroovy
:buildProperties
:processResources
:classes
:findMainClass
:bootRun
Grails application running at http://localhost:8080 in environment: development

> Building 85% > :bootRun

Windows Command Line 

C:\...\helloworld>gradlew bootRun
:compileJava NO-SOURCE
:compileGroovy
:buildProperties
:processResources
:classes
:findMainClass
:bootRun
Grails application running at http://localhost:8080 in environment: development

> Building 85% > :bootRun

Linux Support for Arduino Leonardo / Micro USB Game Controllers

A few months ago I posted an article that described how 3 USB game controllers could be added to the Arduino Leonardo and the Arduino Micro. Out of the box, they look like a USB serial port, keyboard, and mouse to the host computer, but with these modifications they can also look like 3 game controllers. This works great on Microsoft Windows (I have tested it on 7, 8.1, and 10), but it does not work on Linux.

If you plug the modified Arduino Leonardo or Arduino Micro into a machine running Linux and do a listing on the input devices, you will see the Arduino mouse, but not the joysticks.

Before Plugging in the Arduino Leonardo or Micro

After Plugging in the Arduino Leonardo

After Plugging in the Arduino Micro

To get this to work on Linux, you need to adjust a setting on the usbhid driver. This can be done by editing the /boot/cmdline.txt file. Add the following string to the end of the line of text in this file (there should only be one line of text in this file):

For the Arduino Leonardo

usbhid.quirks=0x2341:0x8036:0x040

For the Arduino Micro

usbhid.quirks=0x2341:0x8037:0x040

Explanation

0x2341 is the Vendor ID for Arduino
0x8036 is the Product ID for the Leonardo
0x8037 is the Product ID for the Micro
0x040 indicates the Multiple Input “quirk” should be used (i.e. HID_QUIRK_MULTI_INPUT)

Example

After changing the /boot/cmdline.txt file the Linux machine will need to be rebooted. You will need to have root privileges to edit the /boot/cmdline.txt file. Now when the modified Arduino Leonardo or Arduino Micro is plugged in, three new joysticks will appear.

After Applying Setting – Arduino Leonardo

After Applying Setting – Arduino Micro