IFileStoreService SQL Implmentation (Questions and Working Example)

Hey Joel,

I’m on a roll over here, back again with a question that’s stumping me a bit regarding an implementation of IFileStoreService i’ve written, i’ll share the code in a secondary comment as to not clutter the main issue text up. Just as a precursor, i’ve created an SqlBinaryFilestoreService to store assets inside sql server as byte arrays, since there won’t be a ton of images or documents uploaded, it felt necessary to just store them in the DB and not worry about them being accidentally removed at any point due to a deployment or the likes.

The issue i’m seeing is that DeleteAsync, DeleteDirectoryAsync, ClearDirectoryAsync, ClearContainerAsync are never hit. When i create the image in the admin panel, it uses my Create methods just fine and transposes the stream into a byte array, and getting the image works the same way, the methods get hit. But when i delete an image or asset, it never hits these deletion methods. I should note, i’m using the ImageSharp image asset plugin you provided to handle that side of things.

I even stopped my IFileStoreService implementation from injecting and overriding itself and reverted back to using the built in FileSystemFileStorageService (again using the ImageSharp plugin as well). And to my suprise, not even those files get deleted from the file system when you delete them in the CMS.

Any thoughts? I’d be happy to provide this as an example for those who want to do something similar when it’s all cleaned up and working correctly in the deletion aspect of things)

I’ll post the code below.

Thanks again for all the help and insight!

[mcockrellsana]

EDIT: @HeyJoel Updated with complete FileStoreService for SQL solution, i’m sure it can be cleaned up or optimized a tiny bit, but i’m confident that it works and does what I need it to do.

IFileStoreService Implentation

public class SqlBinaryFilestoreService : IFileStoreService
    {
        // Custom DBContext for use in Conjunction with Cofoundry DBContext
        private readonly WwtlDbContext _wwtlDbContext;
        private readonly ITransactionScopeManager _transactionScopeManager;

        public SqlBinaryFilestoreService(WwtlDbContext wwtlDbContext, ITransactionScopeManager transactionScopeManager)
        {
            // Custom DBContext for use in Conjunction with Cofoundry DBContext
            _wwtlDbContext = wwtlDbContext;

           // Transaction scope manager so we can keep scope down a chain of child scopes for context calls
            _transactionScopeManager = transactionScopeManager;
        }

        /// <summary>
        /// Determines if the specified file exists for the specified container name
        /// </summary>
        /// <param name="containerName">The name of the container to look for the file.</param>
        /// <param name="fileName">Name of the file to look for.</param>
        /// <returns>True if the file exists; otherwise false.</returns>
        public async Task<bool> ExistsAsync(string containerName, string fileName)
        {
            var fileExists = await GetFileAsync(containerName, fileName);

            return fileExists != null;
        }

        /// <summary>
        /// Gets the specified file as a Stream. 
        /// </summary>
        /// <param name="containerName">The name of the container to look for the file</param>
        /// <param name="fileName">The name of the file to get</param>
        /// <returns>Stream reference to the file.</returns>
        public async Task<Stream> GetAsync(string containerName, string fileName)
        {
            var file = await GetFileAsync(containerName, fileName);

            return file != null ? new MemoryStream(file.File) : null;
        }

        /// <summary>
        /// Creates a new file, throwing an exception if a file already exists with the same filename
        /// </summary>
        public Task CreateAsync(string containerName, string fileName, Stream stream)
        {
            return CreateAsync(containerName, fileName, stream, false);
        }

        /// <summary>
        /// Saves a file, creating a new file or overwriting a file if it already exists.
        /// </summary>
        public Task CreateOrReplaceAsync(string containerName, string fileName, Stream stream)
        {
            return CreateAsync(containerName, fileName, stream, true);
        }

        /// <summary>
        /// Creates a new file if it doesn't exist already, otherwise the existing file is left in place.
        /// </summary>
        public Task CreateIfNotExistsAsync(string containerName, string fileName, Stream stream)
        {
            return CreateAsync(containerName, fileName, stream, false);
        }

        /// <summary>
        /// Deletes a file for the container if it exists.
        /// </summary>
        /// <param name="containerName">The name of the container containing the file to delete</param>
        /// <param name="fileName">Name of the file to delete</param>
        public async Task DeleteAsync(string containerName, string fileName)
        {
            var file = await GetFileAsync(containerName, fileName);

            if (file != null)
            {
                try
                {
                    using (var scope = _transactionScopeManager.Create(_wwtlDbContext))
                    {
                        _wwtlDbContext.Set<FileStorage>().Remove(file);
                        await _wwtlDbContext.SaveChangesAsync();
                        await scope.CompleteAsync();
                    }
                }
                catch (Exception ex)
                {
                    throw new InvalidOperationException($"Something went wrong when deleting fileName:{fileName}, containerName:{containerName} in [DeleteAsync] (SqlBinaryFilestoreService)", ex);
                }
            }
        }

        /// <summary>
        /// Deletes a directory including all files and sub-directories
        /// </summary>
        /// <param name="containerName">The name of the container containing the directory to delete</param>
        /// <param name="directoryName">The name of the directory to delete</param>
        public async Task DeleteDirectoryAsync(string containerName, string directoryName)
        {
            try
            {
                var files = new List<FileStorage>();

                using (var scope = _transactionScopeManager.Create(_wwtlDbContext))
                {
                    files = await _wwtlDbContext
                        .Set<FileStorage>()
                        .Where(x => x.Container == containerName && x.FileName.StartsWith(directoryName)).ToListAsync();  

                    await scope.CompleteAsync();
                }

                await DeleteFilesAsync(files);
            }
            catch (Exception ex)
            {
                throw new InvalidOperationException($"Something went wrong deleting sub asset groups containerName:{containerName}, directorName:{directoryName} in [DeleteDirectoryAsync] (SqlBinaryFilestoreService)", ex);
            }
        }

        public Task ClearDirectoryAsync(string containerName, string directoryName)
        {
            return DeleteDirectoryAsync(containerName, directoryName);
        }

        public async Task ClearContainerAsync(string containerName)
        {
            try
            {
                var files = new List<FileStorage>();

                using (var scope = _transactionScopeManager.Create(_wwtlDbContext))
                {
                    files = await _wwtlDbContext
                        .Set<FileStorage>()
                        .Where(x => x.Container == containerName).ToListAsync();

                    await scope.CompleteAsync();
                }

                await DeleteFilesAsync(files);
            }
            catch (Exception ex)
            {
                throw new InvalidOperationException($"Something went wrong deleting sub asset groups containerName:{containerName} in [ClearContainerAsync] (SqlBinaryFilestoreService)", ex);
            }
        }

        private async Task DeleteFilesAsync(IEnumerable<FileStorage> files)
        {
            try
            {
                using (var scope = _transactionScopeManager.Create(_wwtlDbContext))
                {
                    foreach (var fileItem in files)
                    {
                        if (fileItem != null)
                        {
                            _wwtlDbContext.Set<FileStorage>().Remove(fileItem);
                        }
                    }

                    await _wwtlDbContext.SaveChangesAsync();
                    await scope.CompleteAsync();
                }
            }
            catch (Exception ex)
            {
                throw new InvalidOperationException("Something went wrong when deleting multiple files in [DeleteFileAsync] (SqlBinaryFilestoreService)", ex);
            }
        }

        private async Task CreateAsync(string containerName, string fileName, Stream stream, bool overwrite)
        {
            var file = await GetFileAsync(containerName, fileName);

            if (file == null || overwrite)
            {
                try
                {
                    using (var scope = _transactionScopeManager.Create(_wwtlDbContext))
                    {
                        // remove the current file since it exists
                        if (file != null)
                        {
                            _wwtlDbContext.Remove(file);
                        }

                        var addFile = new FileStorage()
                        {
                            Container = containerName,
                            FileName = fileName
                        };

                        using (var memStream = new MemoryStream())
                        {
                            stream.Position = 0;
                            await stream.CopyToAsync(memStream);
                            addFile.File = memStream.ToArray();
                        }

                        // save the new file
                        _wwtlDbContext.Set<FileStorage>().Add(addFile);

                        await _wwtlDbContext.SaveChangesAsync();
                        await scope.CompleteAsync();
                    }
                }
                catch (Exception ex)
                {
                    throw new InvalidOperationException($"Something went wrong when saving the file:{fileName}, containerName:{containerName}, overwrite: {overwrite} in [CreateAsync] (SqlBinaryFilestoreService)", ex);
                }
            }
            else
            {
                throw new InvalidOperationException($"File already exists fileName:{fileName}, containerName:{containerName} in [CreateAsync] (SqlBinaryFilestoreService) ");
            }
        }

        private async Task<FileStorage> GetFileAsync(string containerName, string fileName)
        {
            try
            {
                using (var scope = _transactionScopeManager.Create(_wwtlDbContext))
                {
                    var file = await _wwtlDbContext
                        .Set<FileStorage>()
                        .SingleOrDefaultAsync(x => x.Container == containerName && x.FileName == fileName);

                    await scope.CompleteAsync();

                    return file;
                }
            }
            catch (Exception ex)
            {
                throw new InvalidOperationException($"Something went wrong when retrieving the fileName:{fileName}, containerName:{containerName} in [GetFileAsync] (SqlBinaryFilestoreService)", ex);
            }
        }
    }

Custom DB Context and Entity Initialization

public class FileStorage
    {
        public string FileName { get; set; }
        public string Container { get; set; }
        public byte[] File { get; set; }
    }

public class WwtlDbContext : DbContext
    {

        private readonly ICofoundryDbContextInitializer _cofoundryDbContextInitializer;

        public WwtlDbContext(ICofoundryDbContextInitializer cofoundryDbContextInitializer)
        {
            _cofoundryDbContextInitializer = cofoundryDbContextInitializer;
        }

        protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
        {
            _cofoundryDbContextInitializer.Configure(this, optionsBuilder);
        }

        protected override void OnModelCreating(ModelBuilder modelBuilder)
        {
            base.OnModelCreating(modelBuilder);

            modelBuilder
                .ApplyConfiguration(new FileStorageConfiguration());
        }

    }

public class FileStorageConfiguration : IEntityTypeConfiguration<FileStorage>
    {
        public void Configure(EntityTypeBuilder<FileStorage> builder)
        {
            builder.ToTable("FileStorage", "dbo");

            builder.HasKey(k => new {k.FileName, k.Container});

            builder.Property(p => p.Container)
                .IsRequired()
                .IsUnicode()
                .HasMaxLength(250);

            builder.Property(p => p.File)
                .IsRequired();

            builder.Property(p => p.FileName)
                .IsRequired()
                .IsUnicode()
                .HasMaxLength(500);
        }
    }

Cofoundry DI For Custom DBContext

public class DataDependencyRegistration : IDependencyRegistration
    {
        public void Register(IContainerRegister container)
        {
            container.RegisterScoped<WwtlDbContext>();
        }
    }

[HeyJoel]

Thanks for the note about the HangFire upgrade, I had seen that there’s a tricky migration to do for 1.7 but had not seen anything about the cron expressions, I’ve now added an issue for it on that repo.

I’ll leave this open as a reminder to update the docs for the AssetFileCleanupSettings.