Sunday, June 28, 2020

TypeScript hasDuplicates Function

Here is a simple TypeScript function that can be used to determine if a strongly typed TypeScript array contains duplicate items:

function hasDuplicates<T>(inputArray: T[]): boolean {
    return inputArray.length !== (new Set(inputArray)).size;
}

Example usage:

Sunday, May 24, 2020

Unicode Filename Support in TAR Files in Windows 10

No Unicode Filename Support in ZIP Files in Windows 10

I recently discovered that the built-in ZIP file support in Windows 10 does not support Unicode filenames. For example, I had a file name “みんな一列に.jpg” and tried to add it to a ZIP file, but got the following error:

No Unicode Filename Support in TAR Files in Windows 10

Windows 10 has added built-in support for TAR files through a command line tool. Since I was unable to put files with Unicode filenames into ZIP files in Windows, I tried using this tool to put them into a TAR file. I used the following command to add the “みんな一列に.jpg” file to a new TAR file called test.tar:
D:\Temp>dir /b *.jpg
みんな一列に.jpg

D:\Temp>tar -cvf test.tar *.jpg
a ??????.jpg
Unfortunately, it replaced the Unicode characters with question marks (?). This replacement of characters can also be seen when performing the following command:
D:\Temp>tar -tvf test.tar
-rw-rw-rw-  0 0      0       37044 Nov 10  2004 ??????.jpg

WSL to the Rescue

Windows 10 has the ability to run UNIX command line tools using the Windows Subsystem for Linux. See https://docs.microsoft.com/en-us/windows/wsl/install-win10 for instructions on how to install the Windows System for Linux. Linux also contains a command line tool for creating TAR files. I attempted to create a TAR file containing the “みんな一列に.jpg” file using the following command in an Ubuntu bash shell:
matthew@KIKI2015:/mnt/d/Temp$ ls *.jpg
みんな一列に.jpg
matthew@KIKI2015:/mnt/d/Temp$ tar -cvf test.tar *.jpg
みんな一列に.jpg
I first verified the filename was preserved using the following command:
matthew@KIKI2015:/mnt/d/Temp$ tar -tvf test.tar
-rwxrwxrwx matthew/matthew 37044 2004-11-10 00:54 みんな一列に.jpg
I then extracted the file to a new folder using the following command:
matthew@KIKI2015:/mnt/d/Temp$ cd test
matthew@KIKI2015:/mnt/d/Temp/test$ tar -xvf ../test.tar
みんな一列に.jpg
It even appeared correctly in Windows File Explore:

Back to Windows 10

I was curious to see how the Windows 10 tar program would react to the TAR file created in Linux, so I tried listing the contents of the TAR file using a Windows 10 command prompt window:
D:\Temp>tar -tvf test.tar
-rwxrwxrwx  0 matthew matthew 37044 Nov 10  2004 pü+péôpü¬S+Çsêùpü½.jpg
Unsurprisingly it did not interpret the filename correctly. Interestingly, extract the file resulted in yet a different filename: みんな一列に.jpg.

Hopefully Microsoft will add support for Unicode filenames in ZIP or TAR files in a future update to Windows, but until then, WSL can be used.

Sunday, November 25, 2018

Combining mp4 Files Without Re-encoding Using FFmpeg on Windows Subsystem for Linux

I recently used FFmpeg installed on the Windows Subsystem for Linux on my Windows 10 machine to combine some mp4 video files without re-encoding the files. Being able to do this without re-encoding makes the process much quicker and does not result in image degradation.

Install the Windows Subsystem for Linux

If it is not already installed, you will need to install the Windows Subsystem for Linux. Instructions for doing this can be found at https://docs.microsoft.com/en-us/windows/wsl/install-win10. I have the Ubuntu distro installed (https://www.microsoft.com/en-us/p/ubuntu/9nblggh4msv6).

Install FFmpeg

To install FFmpeg (https://ffmpeg.org/) on Ubuntu open the Ubuntu command line and enter the following:
sudo apt install ffmpeg

Create Merge File

Using the text editor of your choice, create a text file that contains the list of mp4 files you want to combine. For this example I called the file input.txt. The format of this file is as follows:
file 'part1.mp4'
file 'part2.mp4'

Combine mp4 Files

The following command can be used to merge the files listed in the text file from the step above into a single mp4 output file without re-encoding the files:
ffmpeg -f concat -i input.txt -c copy output.mp4
The name of the new mp4 file in the example above is output.mp4.