WSL2 Extremely slow file read fix (mutagen)

Current Env:

Windows 10

WSL2 unbuntu 20.04

This document is intended to solve the WSL2 slow file read issue via using a docker container bridge to construct a forward file sync from Windows host to docker container volume mirror mount to the WSL2 container.

The development of IDE env will be hosted on Windows session, not WSL2 session.

The hosting / script / test session can be on both Windows / WSL2 session.

The pros and cons of using this method:

Pros:

  • solve WSL2 slow file read issue with text editor
  • almost native file read speed
  • At least it is a alternative fix for a 2 yrs issue
  • You dont need a linux ext4 drive for this

Cons:

  • requires docker container volume bridge running during the session
  • some of the binaries may not be processed or run because of the different host issue, (node_modules bin), you need to setup a custom file sync blacklist and config the file on WSL2 side
  • need to run dos2unix if you are running .sh files on wsl2 when writing it from windows

Windows:

  1. Download mutagen to your windows, I am using mutagen v0.11.8 here , https://github.com/mutagen-io/mutagen/releases
  2. Config windows path for mutagen (copy to C:.bin, than add C:.bin to sys env path)
  3. Open Powershell session, test your if mutagen is working corectly,
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
> mutagen

PS C:\Users\Engineer> mutagen
Mutagen is a remote development tool built on high-performance synchronization

Usage:
mutagen [flags]
mutagen [command]

Available Commands:
sync Create and manage synchronization sessions
forward Create and manage forwarding sessions [Experimental]
project Orchestrate sessions for a project [Experimental]
tunnel Create and manage tunnels [Experimental]
login Log in to mutagen.io
logout Log out from mutagen.io
daemon Control the lifecycle of the Mutagen daemon
version Show version information
legal Show legal information
help Help about any command

Flags:
-h, --help Show help information

Use "mutagen [command] --help" for more information about a command.

On WSL2:

  1. goto ~/ , setup mirror path
1
2
3
4
5
6
7
8
9
10
11
12
13
cd ~
mkdir docker-mirrors
cd docker-mirrors

# remove UBUNTU_BRIDE container if exists
docker rm UBUNTU_BRIDGE
# start the session
docker run -it -v $PWD:/app --name UBUNTU_BRIDGE ubuntu:20.04 /bin/bash

# In the docker session
# config a user with the same user id of WSL2 , default is 1000
adduser ubuntu
cd /app # here are the synchronized files

Setup development environment on Windows

  1. goto %HOMEPATH%, default is C:\Users\{YOUR_USERNAME}\
  2. create a default .mutagen.yml, (there is . before the mutagen.yml, make sure you have it), for ignoring node_modules and other files that should only be working on the host (for example linux based binaries)
1
2
3
4
5
6
7
8
9
10
11
12
13
# write this file at %HOMEPATH%.mutagen.yml

sync:
defaults:
ignore:
paths:
- "mutagen.yml"
- "mutagen.yml.lock"
- ".vagrant"
- ".git"
- ".idea"
- "node_modules"
vcs: true

Setup Project file sync on Windows (You only need to do step 7-10 for new projects)

  1. goto your project root path, and start the mutagen session:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
> mutagen sync create . docker://ubuntu@UBUNTU_BRIDGE/app/project_name
Created session sync_Gpwdlf1Dor5PEOtkNBBzZYN1LXci6hkl7AZzJPVZavp

> mutagen sync list sync_Gpwdlf1Dor5PEOtkNBBzZYN1LXci6hkl7AZzJPVZavp
Identifier: sync_Gpwdlf1Dor5PEOtkNBBzZYN1LXci6hkl7AZzJPVZavp
Labels: None
Alpha:
URL: C:\Users\Engineer\Projects\xxx\yyy
Connection state: Connected
Beta:
URL: docker://ubuntu@UBUNTU_BRIDGE/app/project_name
DOCKER_HOST=
DOCKER_TLS_VERIFY=
DOCKER_CERT_PATH=
DOCKER_API_VERSION=
Connection state: Connected
Status: Watching for changes
  1. On your docker mirror session, verify the sync
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
root@29dfaab60681:/app# ls -la project_name/
total 236
drwx------ 8 ubuntu ubuntu 4096 Sep 30 01:19 .
drwxr-xr-x 4 ubuntu ubuntu 4096 Sep 30 01:19 ..
-rw------- 1 ubuntu ubuntu 141 Sep 30 01:19 .editorconfig
drwx------ 7 ubuntu ubuntu 4096 Sep 30 01:19 .git
-rw------- 1 ubuntu ubuntu 84 Sep 30 01:19 .gitignore
drwx------ 2 ubuntu ubuntu 4096 Sep 30 01:19 .idea
-rw------- 1 ubuntu ubuntu 345 Sep 30 01:19 README.md
-rw------- 1 ubuntu ubuntu 229 Sep 30 01:19 _config.yml
-rw------- 1 ubuntu ubuntu 942 Sep 30 01:19 package.json
drwx------ 2 ubuntu ubuntu 4096 Sep 30 01:19 scaffolds
drwx------ 5 ubuntu ubuntu 4096 Sep 30 01:19 source
drwx------ 3 ubuntu ubuntu 4096 Sep 30 01:19 themes
-rw------- 1 ubuntu ubuntu 171151 Sep 30 01:19 yarn.lock
root@29dfaab60681:/app#
  1. On your WSL2, verify the syncS
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
> ~/projects/docker-mirrors# ls -la project_name
total 236
drwx------ 8 engineer engineer 4096 Sep 30 09:19 .
drwxr-xr-x 4 engineer engineer 4096 Sep 30 09:19 ..
-rw------- 1 engineer engineer 141 Sep 30 09:19 .editorconfig
drwx------ 7 engineer engineer 4096 Sep 30 09:19 .git
-rw------- 1 engineer engineer 84 Sep 30 09:19 .gitignore
drwx------ 2 engineer engineer 4096 Sep 30 09:19 .idea
-rw------- 1 engineer engineer 345 Sep 30 09:19 README.md
-rw------- 1 engineer engineer 229 Sep 30 09:19 _config.yml
-rw------- 1 engineer engineer 942 Sep 30 09:19 package.json
drwx------ 2 engineer engineer 4096 Sep 30 09:19 scaffolds
drwx------ 5 engineer engineer 4096 Sep 30 09:19 source
drwx------ 3 engineer engineer 4096 Sep 30 09:19 themes
-rw------- 1 engineer engineer 171151 Sep 30 09:19 yarn.lock

# you can now run all your dev scripts here:
> yarn
> ls -la .
total 232
drwx------ 6 engineer engineer 4096 Sep 30 09:28 .
drwxr-xr-x 4 engineer engineer 4096 Sep 30 09:25 ..
-rw------- 1 engineer engineer 141 Sep 30 09:25 .editorconfig
-rw------- 1 engineer engineer 84 Sep 30 09:25 .gitignore
-rw------- 1 engineer engineer 345 Sep 30 09:25 README.md
-rw------- 1 engineer engineer 229 Sep 30 09:25 _config.yml
drwxr-xr-x 484 engineer engineer 20480 Sep 30 09:29 node_modules
-rw------- 1 engineer engineer 942 Sep 30 09:25 package.json
drwx------ 2 engineer engineer 4096 Sep 30 09:25 scaffolds
drwx------ 5 engineer engineer 4096 Sep 30 09:25 source
drwx------ 3 engineer engineer 4096 Sep 30 09:25 themes
-rw------- 1 engineer engineer 171151 Sep 30 09:25 yarn.lock

# you may have fun now
# everything will work as normal, including browsersync etc.
> yarn server
  1. Once completed the session, terminate it on Powershell
1
2
3
4
mutagen sync terminate sync_Gpwdlf1Dor5PEOtkNBBzZYN1LXci6hkl7AZzJPVZavp
mutagen sync terminate sync_Gpwdlf1Dor5PEOtkNBBzZYN1LXci6hkl7AZzJPVZavp

mutagen sync terminate sync_Gpwdlf1Dor5PEOtkNBBzZYN1LXci6hkl7AZzJPVZavp

Bye , see you guys around.