Data Extract Missing Data Content & Extracted Total does not Align

[TSHECKEL]

I ran across an issue whereas the data extract for a table was missing records and, in turn, the extracted total did not equal the total within the visualization. After a bit of analysis, it appears that there's an issue with the extract utility when it comes to aggregating and interpreting like dimensional values. In my case, the extract dropped records when upper and lowercase like values existed. 

I was able to resolve this by setting the Attribute 2 field to all uppercase within the transform settings and refreshing the data content. Consequent data extracts produced all records and the total aligned. 

I would have expected the data extract to either 1. treat these all as unique rows or 2. combine the like rows regardless of case. However, a bug exists whereas the data extract is dropping records. 

[v-cazheng-msft]

Hi @TSHECKEL 

May I know whether your issue is that not all the rows be displayed in your visual? If you add an index row in Power Query, will you get same issue?

Data displayed in Data View:

Data displayed in the Table visual:

If I misunderstand your problem, could you please make it more clear with some screenshots or a link of sample pbix file containing this issue? In addition, please let me know these info.

• What’s your data source? Is it a Excel workbook?
• What connector did you use to connect to your data source?
• The version of your Power BI Desktop. You can check it by Help>About>version.

Best Regards,

Community Support Team _ Caiyun

[TSHECKEL]

Thanks for looking at this and replying. This dataset is promoted as a shared dataset to our internal analyst community and we're a Premium subscriber. 

Answers to Questions

• The rows are also missing from within the visualization(s). 
• If I import the example I provided through the use of Excel, the scenario does not exist and the rows are aggreagted based on one or the other, upper or lowercase, dimensions.
• All of the data content is pulled in from an underlying Snowflake DB
• We're utilizing the Power BI prebuilt Snowflake API connector (straight extracts with no embedded scripts)
• PBI Desktop Version: 2.100.1401.0 64-bit (December 2021)

Storage Modes

• Attribute 1 = Dual 
• Attribute 2 = Dual
• Attribute 3 = Direct Query
• Attribute 4 = Direct Query
• Attribute 5 = Direct Query

We have numerous data feeds that flow into our source DB. With this being the case, I don't believe that setting an indexed column would be efficient or even possible. This would have to be put in place for tons of fields. Also, this scenario of upper vs lowercase could flow in at any time for any given text field. 

I have verified that the total amount and / or counts is displayed correctly within the visualization. However, the visualization and export utility are not displaying all rows and the sum of these does not equal the visualization totals.

Please let me know if there's anything else I can provide or assist with in regards to this issue. 

[TSHECKEL]

I ran another test by taking the dataset model, taking a small subset of the data and switching it to a full import storage mode. The visualization then rendered all rows and the extracts total aligned with the visualizations reported total. However, a full import storage mode is not feasible due to the underlying volume and overall size of this dataset.

[TSHECKEL]

@v-cazheng-msft Hi, I was curious as to if there was any further resarch being conducted on this issue or if there's any other questiones for me or anything else needed from my end. This issue is still present with the Dual connection feature. Thanks

[TSHECKEL]

@v-cazheng-msft This issue has presented itself again within our analayst community. Has there been any further research or insight conducted on this bug? Thanks

[TSHECKEL]

@v-cazheng-msft Has there been any further research or insight conducted on this bug? Thanks

[TSHECKEL]

@v-cazheng-msft Following up to see if there's been any further research on this issue. Thanks

[TSHECKEL]

@v-cazheng-msft Should I duplicate this scenario elsewhere in the forum? It doesn't seem to be gaining any traction and this issue is still present today. Thanks! 

[TSHECKEL]

Any updates on this scenario? This is still present.