Alluxio master Metaspace OOM

Alluxio Version:

Master branch

Describe the bug

• Master down
• master.out

java.lang.OutOfMemoryError: Metaspace
Dumping heap to /data/alluxio/alluxioenv/alluxio-logs/logs/master.hprof ...
Heap dump file created [6770171579 bytes in 46.476 secs]
Terminating due to java.lang.OutOfMemoryError: Metaspace

For now, mount 60 hdfs ufs can use 850MB metaspace.

To Reproduce

bin/alluxio fs mount /hdfs1 hdfs://localhost:1111/
jcmd <PID> VM.metaspace 

bin/alluxio fs mount /hdfs2 hdfs://localhost:1112/
jcmd <PID> VM.metaspace 

bin/alluxio fs mount /hdfs3 hdfs://localhost:1112/
jcmd <PID> VM.metaspace 
...

Expected behavior

The metaspace used increase a small size.

Urgency

Urgent, it limit the count of mountpoint for our scenario.

Are you planning to fix it

Not yet.

Additional context

We should support 1k mountpoint with few of metaspace used.

A possible approach: put the common dependencies into a jar or classpath, use a customized parent classloader to load these class first.

[LuQQiu]

https://github.com/Alluxio/alluxio/pull/16145 @Haoning-Sun has a PR to fix it

[maobaolong]

It will increase 10M+ after mount a hdfs mount point, this can limit the count of mount point.