Windows 2016 Deduplication on FSLogix Containers and ODFC – Part 1

By Ray Davis posted 03-04-2021 08:47 AM

  

Windows 2016 Deduplication on FSLogix Containers and ODFC 

During the time of running my Shrink Scripts (yes, I have a couple I go back and forth with) for FSLogix, I noticed that my 4TB drive is used more and more. There wasn’t really anything I could do at this point other than adding disk space (call me old, but I try to optimize as much space as possible before throwing more hardware at it). One of the things I could do is delete all older profiles that haven’t been used in 90 days. Some scripts can do this for you. One is by Jim Moyle, and another is by René Bigler (posted at the end). However, I must test this, and I don’t have the time to do this right now. So, Manuel Winker reminded me about Windows Deduplication. I have totally forgotten about this, and I am not sure how I did, to be honest. So, this is how I did it and my results.

  1. Install the Data Deduplication Role (GUI)

Install the Data Deduplication Role (GUI)


  1. PowerShell install: Install-WindowsFeature -Name FS-Data-Deduplication -IncludeAllSubfeature -IncludeManagementTools 

PowerShell Install


  1. After things are installed, you need to set it up for the Volume that you will apply this to.
  2. Open server manager, then navigate to File and Storage Services --> Volumes, select the desired volume, right-click and from the menu and select Configure Data Deduplication.

Configure Data Deduplication


For this test, I am going to use the “General Purpose File server.”

This is a chart I pulled from the MS site on the difference: https://docs.microsoft.com/en-us/windows-server/storage/data-deduplication/understand#usage-type-default


Data Deduplication Config Chart


Deduplication Settings


For the Schedule, I had some advice from Mike Streetz: https://worldofeuc.slack.com/archives/CKBVDG48H/p1610685976025900

I removed Enable Background Optimizations because I felt that it could cause issues, and enabled a schedule based on some feedback with better control.

Remove background optimization


Let’s check my schedule with PowerShell: Get-DedupSchedule

PowerShell Schedule Check


I am going to kick it off and see what it does for me.

Powershell: Start-DedupJob -Volume D: -Type Optimization -Memory 75 -Cores 100 -Priority High -Full

Start PowerShell Job


Before:

Before Deduplication


After:

After Deduplication


Let check the status: Get-DedupStatus -Volume D: | fl

Status Check


So, I save 1GB? WOW. After looking at this, I realized that I ran an FSLogix Shrink script 2 days ago. So, I needed to change it to 1 day for me. So, that was not a really fair test.

Test 2 is switching to VDI and 1 day. Based on what I read on MS, this would help in saving around seeing the same OS over and over. But in this case, it’s just a test, and I am not worried about that. 

VDI Setting


After:

After Deduplication VDI


Dedup Status


So, I saved 92.71 GB.

I will switch it back to see if I can squeeze more out or if it better this way. But I doubt it.


FSLogix Profiles


So, I'll run this again.

Start-DedupJob -Volume D: -Type Optimization -Memory 75 -Cores 100 -Priority High -Full 

Results are the same. But this was a good experiment, and if I saved 91GB on 500GB Volume, I gained 18.5% of disk space back. 500×18.2% = 92.5

What could I save on 4TB? If we take the math I have here and just guesstimate (guessing and estimating): 4000 x 18.5% = 740GB. I could gain another 740GB. At 4TB volume, my current free space is 670GB after an FSLogix Shrink job was run. So, overall, that is 1.4TB back in my pocket. I call that a win and I'm very happy. However, this could be less or more. 

In my opinion, this is a huge value to anyone running FSLOGIX Profile servers, or really any file servers. This quick blog is not fancy, but more of a reminder out there for those who forgot like me. Everyone be safe and until next time, folks.


Resources

https://worldofeuc.slack.com/archives/CKBVDG48H/p1610685976025900

https://docs.microsoft.com/en-us/windows-server/storage/data-deduplication/understand

https://docs.microsoft.com/en-us/windows-server/storage/data-deduplication/advanced-settings


Cleanup Jobs

Cleanup unused FSLogix Office 365 Containers – dready’s Blog

Delete Inactive FSLogix Profiles using PowerShell – hallspalmer_Blog (wordpress.com)


Delete old Profiles script

GitHub – FSLogix/Invoke-FslShrinkDisk: This script will shrink a FSLogix Disk to its minimum possible size

#Windows Deduplication with FSLogix
#FSLOGIX

​​​
#FSLogix
#Windows_Deduplication_FSLogix

Comments

09-15-2021 05:31 PM

Just wondering if this is still working for you?  Any impact on speed/IO jbythrow@presidio.com