Semantic Model - Direct Lake not refreshing

[YOUNG99]

Trying to figure out if I'm misunderstanding the concept, misconfigured the setup, or Fabric not functioning as expected.

My understanding is that Semantic Model Storage Direct Lake works like Direct Query where the latest data is shown in the Power BI without a need to schedule a refresh of the Semantic Model.  I have the below setup and I see stale data or an error. 

1.  Using Power BI P1 premium workspace
2.  Created a Lakehouse in the above workspace

3.  Created and scheduled a pipeline, that overwrites a table in the ablove Lakehouse every 5 minutes

4.  Create a semantic model with Direct Lake Storage Mode from the above table. Verified this is toggled on:  Keep your Direct Lake data up to date
5.  Create a Power BI report using the semantic model above.

Observed:
A.  All objects created was under same account with Capacity Admin rights.
B.  Verifed the underlying table is refreshed every 5 minutes in the Lakehouse
C.  Encountered the following error on occassion:  Unexpected parquet exception occurred. Class: 'ParquetStatusException' Status: 'IOError' Message: 'Encountered Azure error while accessing lake file, StatusCode = 404, ErrorCode = BlobNotFound, Reason = Not Found'
D.  Power BI is showing stale data, but has randomly shown newer data at least once.  (after encountering the above error)

Thank You

"Unexpected parquet exception occurred. Class: 'ParquetStatusException' Status: 'IOError' Message: 'Encountered Azure error while accessing lake file, StatusCode = 404, ErrorCode = BlobNotFound, Reason = Not Found'"

[YOUNG99]

@v-rongtiep-msft  thanks for the link.  I couldn't apply the solution as I only had a single table.

 
The issue seemed to have resovled itself however.  I even ramped up the Pipeline to every 1 minute and the Power BI report is refreshing as expected. 

On very rare occasions I do observe the following:
1. (blank)  from visuals
2.  or an error stating it couldn't find the table in Fabric

I'm using override data instead of append so I think the above happens when it's copying over new data and resolving between the temp table  "<tablename>_backup_<GUID>" and the real table.  For production I may have to resort to append and add a step in the pipeline to delete the stale data.  Unless there are better suggestions?

Thanks!  

[v-rongtiep-msft]

Hi @YOUNG99 ,

I found a post here with a similar discussion and you can see if it helps you.

Unexpected parquet exception occurred. Class: 'Par... - Page 2 - Microsoft Fabric Community

If it does not help, please provide more details.

Best Regards
Community Support Team _ Rongtie

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[BiJoe]

@v-rongtiep-msft What do you mean by "see if this helps you"? In the thread you are linking to, the OP says "maybe this works but clearly this should be a top issue to solve for the fabric team..", and there are 5 users asking for updates.

I myself experience this regularly.

Please inform us if this is a known Fabric issue, and what Microsoft is doing to fix this.