Improving the .NET API to work with the structure of the file system. Part 2. Manipulate filesystem objects.

In the first part of the article, I described improved methods to enumerate file system objects. In this post, I will describe methods for manipulating them.

I note that there is a new possibility to enumerate file system objects. The FileSystemTraversalOptions enumerated type has three new flags: ShowFileOccurrence, ShowDirectoryEnter, ShowDirectoryLeave and their combinations. These flags affect the results of methods that enumerate file system objects of all types or partially for directories. The ShowFileOccurrence flag specifies the output of file objects. ShowDirectoryEnter specifies the output of a directory when entering it from its parent directory. The ShowDirectoryLeave flag sets the output of the directory when leaving it to the parent directory. It is easy to see that the combination of the ShowFileOccurrence and ShowDirectoryEnter flags specifies a preorder traversal (PreorderTraversal) of the file system object tree, while the combination of the ShowFileOccurrence and ShowDirectoryLeave flags specifies a postorder traversal (PostorderTraversal). It is also possible to combine the ShowDirectoryEnter and ShowDirectoryLeave (ShowDirectoryOccurrence) flags, which specifies the appearance of one directory twice: when entering it from the parent and when leaving it to the parent. Using methods with custom results, you can mark the output of file system elements with a FileSystemTraversalMarker enumerated value with values: None, EnterDirectory, LeaveDirectory using the ToMarkedInfo extension method on an object of type derived from FileSystemInfo resulting in an object of type MarkedInfo<TInfo>. Below is a code snippet to display the full path of files and subdirectories from all levels.

new DirectoryInfo(@"C:\Temp")
  .EnumerateFileSystemInfos(FileSystemTraversalOptions.ShowAllOccurrence,
    (fileSystemInfo, traversalMarker) => fileSystemInfo.ToMarkedInfo(traversalMarker))
  .ForEach(item => Console.WriteLine($"{item.Info.FullName} : {item.TraversalMarker}"));

Let's now move on to methods for manipulating a group of file system objects. There are both versions of methods with custom functionality specified by the calling method and methods that implement the following predefined operations: copying, moving, replacing and deleting files, moving, deleting, replicating (copying structure) and clearing (deleting empty) directories.
Each group can have up to 3 types of method implementations: regular synchronous, asynchronous (an asynchronous call to a synchronous method using a thread pool is implemented for each file system object) and parallel synchronous using PLINQ to parallelize work with files, but not with directories, which are performed sequentially. All input methods accept enumerations of both the file system objects of the FileSystemInfo, FileInfo, DirectoryInfo types and custom containers containing them.
Asynchronous methods (have the Async suffix) return processing results through the IAsyncEnumerable interface. For synchronous methods, there are both implementations with deferred processing (they have the Defer suffix) and direct processing of all elements. Elements in deferred processing methods are processed as they are retrieved by external code from the enumeration. Methods with direct processing return the result of processing all elements. As I said before, methods with parallel processing capability only process file objects in parallel. Such methods have an additional maxConcurrents parameter that specifies the maximum degree of parallelism. Below is a table of testing methods for element-by-element deleting and copying a directory with a 13 GB project containing about 200,000 files and 33,000 directories. The time of work with a local and network drive (WiFi connection) was measured. After each test, the pages that cache files were reset.

Element-by-element deletion.

Element-by-element copying.

Each method has a FileSystemManipulationOptions enumerated parameter that contains the following flags to control processing actions:
EnsureDirectory - controls the creation of directories required for the operation. An operation's options type may implement the IEnsuringOptions interface, which must provide a list of required paths that will be created if none are present. This preprocessing is performed only for the directory marked with the value FileSystemTraversalMarker.EnterDirectory, i.e. before processing its child objects.
CleanupDirectory - manages the removal of empty directories. This post-processing is performed only for the directory marked with the value FileSystemTraversalMarker.LeaveDirectory, i.e. after processing all of its child objects.
ClearReadOnly - Controls the clearing of the ReadOnly attribute for file system objects that are about to be deleted or overwritten.
Refresh - controls how the file system object's state is updated before it is used.
SkipEmptyDirectory - skips any processing for empty directories.

Each method also contains an optional delegate for handling errors when working with file system objects - errorHandler. As input, it takes the original object, the operation with which caused an error, the parameters of the operation, and the exception that occurred. It can be used for logging. This delegate must return a boolean value that controls the suppression of the thrown exception and the continuation of processing of subsequent elements.

Let's look at calling methods of 6 operations (copy, move, replace, delete, replicate and cleanup) working on a directory containing files and subdirectories.

1. Copy

The following code causes the entire structure (files and subdirectories) of the srcDirectoryName directory to be copied to the dstDirectoryName directory.

new DirectoryInfo(srcDirectoryName)
  .EnumerateFileSystemInfos(FileSystemTraversalOptions.PreorderTraversal)
  .CopyFileSystemInfos(FileSystemManipulationOptions.EnsureDirectory | FileSystemManipulationOptions.ClearReadOnly,
    fileSystemInfo => new TransferOptions
    {
      DestinationPath = Path.Combine(dstDirectoryName, PwrPath.GetRelativePath(srcDirectoryName, fileSystemInfo.FullName)),
      Overwrite = true,
      NoProcessing = fileSystemInfo is DirectoryInfo,
    });

Files are copied element by element. Non-existent directories in the destination will be created automatically according to the EnsureDirectory flag set. If the copied files exist at the destination, they will be overwritten according to the Overwrite option set in the TransferOptions operation's parameter structure. If existing files to be overwritten have the ReadOnly attribute set, then it will be cleared according to the ClearReadOnly flag set.

2. Move

The following code causes the entire structure (files and subdirectories) of the srcDirectoryName directory to be moved to the dstDirectoryName directory.

new DirectoryInfo(srcDirectoryName)
  .EnumerateFileSystemInfos(FileSystemTraversalOptions.ShowAllOccurrence, (fileSystemInfo, traversalMarker) => fileSystemInfo.ToMarkedInfo(traversalMarker))
  .MoveFileSystemInfos(FileSystemManipulationOptions.EnsureDirectory | FileSystemManipulationOptions.CleanupDirectory | FileSystemManipulationOptions.ClearReadOnly,
    (fileSystemInfo, traversalMarker) => new TransferOptions
    {
      DestinationPath = Path.Combine(dstDirectoryName, PwrPath.GetRelativePath(srcDirectoryName, fileSystemInfo.FullName)),
      Overwrite = true,
      NoProcessing = fileSystemInfo is DirectoryInfo,
    });

Files are moved element by element. Non-existent directories in the destination will be created automatically according to the EnsureDirectory flag set. Empty directories in the original location will be removed according to the CleanupDirectory flag set. If files to be moved exist at the destination, they will be overwritten according to the Overwrite option set in the TransferOptions operation's parameter structure. If existing files to be overwritten or directories to be deleted have the ReadOnly attribute set, then it will be cleared according to the ClearReadOnly flag set.

3. Replacement

The following code causes the files in the dstDirectoryName directory to be replaced with the corresponding files in the srcDirectoryName directory, and optionally to create a backup of the replaced files in the bakDirectoryName directory, creating the required subdirectory structure in it.

new DirectoryInfo(srcDirectoryName)
  .EnumerateFileSystemInfos(FileSystemTraversalOptions.ShowAllOccurrence, (fileSystemInfo, traversalMarker) => fileSystemInfo.ToMarkedInfo(traversalMarker))
  .ReplaceFileSystemInfos(FileSystemManipulationOptions.EnsureDirectory | FileSystemManipulationOptions.CleanupDirectory | FileSystemManipulationOptions.ClearReadOnly,
    (fileSystemInfo, traversalMarker) => new ReplaceOptions
    {
      DestinationPath = Path.Combine(dstDirectoryName, PwrPath.GetRelativePath(srcDirectoryName, fileSystemInfo.FullName)),
      DestinationBackupPath = Path.Combine(bakDirectoryName, PwrPath.GetRelativePath(srcDirectoryName, fileSystemInfo.FullName)),
      NoProcessing = fileSystemInfo is DirectoryInfo,
    });

Files are replaced and backed up element by element. Non-existent directories in the backup location will be created automatically according to the EnsureDirectory flag set. Empty directories in the original location will be removed according to the CleanupDirectory flag set. If existing files in the srcDirectoryName or dstDirectoryName directories or subdirectories of srcDirectoryName to be deleted have the ReadOnly attribute set, then it will be cleared according to the ClearReadOnly flag set.

4. Deletion.

The following code calls for element-by-element deletion of objects in the srcDirectoryName directory.

new DirectoryInfo(srcDirectoryName)
  .EnumerateFileSystemInfos(FileSystemTraversalOptions.PostorderTraversal)
  .DeleteFileSystemInfos(FileSystemManipulationOptions.CleanupDirectory | FileSystemManipulationOptions.ClearReadOnly,
    fileSystemInfo => new DeleteOptions
    {
      NoProcessing = fileSystemInfo is DirectoryInfo,
    });

Files are deleted element by element. Empty directories in the original location will be removed according to the CleanupDirectory flag set. If existing files or directories to be deleted have the ReadOnly attribute set, then it will be cleared according to the ClearReadOnly flag set.

5. Replication.

The following code causes a copy of the subdirectory structure from the srcDirectoryName directory to the dstDirectoryName directory to be created.

new DirectoryInfo(srcDirectoryName)
  .EnumerateDirectories(FileSystemTraversalOptions.PreorderTraversal)
  .ReplicateDirectories(FileSystemManipulationOptions.None,
    directoryInfo => new ReplicateOptions
    {
      DestinationPath = Path.Combine(dstDirectoryName, PwrPath.GetRelativePath(srcDirectoryName, directoryInfo.FullName)),
    });

6. Cleanup.

The following code causes empty subdirectories of the srcDirectoryName directory to be deleted.

new DirectoryInfo(srcDirectoryName)
  .EnumerateDirectories(FileSystemTraversalOptions.PostorderTraversal)
  .CleanupDirectories(FileSystemManipulationOptions.None);

Extended examples of using methods with progress can be found in the FileSystemExamples class.

I note that the nuget package that contains the FileSystemInfoExtension class from namespace PowerLib.System.IO is called VasEug.PowerLib.System and has a MIT license.

P.S. If there is interest, I will probably plan to publish in the future a microservice that provides for each identity registered in it the allocation of its own virtual file space (empty or with a predefined structure) with the ability to upload and download files, copy, move, delete and share files and subdirectories, restrict rights and quotas.