I am quite sure that this is happening because of pivot tables. When you create a PivotTable, Microsoft Excel creates a hidden copy (cache) of the source data that it uses to perform aggregations and other PivotTable functions. By using this cache, Excel can quickly calculate PivotTables, while maintaining the integrity of the original data.
This cache can affect memory use, speed, and file size, depending on the options that you select when you create your PivotTable.