Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn a 50% discount on the DP-600 certification exam by completing the Fabric 30 Days to Learn It challenge.

Reply
DebbieE
Community Champion
Community Champion

Fabric Pyspark confusion. saveastable and parquet

Ive spent hours reading all the documentation and watching the videos and Im now more confused than ever. 

 

I have created a data frame and i basically want to save it as Delta Parquet. im assuming it goes in here

Tables.png

 I ran this code

from delta.tables import DeltaTable
dfcont.write.mode("overwrite").format("delta").saveAsTable("dimContestant")
 
And it created this 
 
DeltaPARQUET.png
 
But then i got confused and ran this (Because I wanted the Delta PARQUET in a folder
 
dfcont.write.mode("overwrite").parquet('Files/Data/Silver/DimContestant.parquet')
 
And it created this
BronzeSilver.png
DimContestant.parquet.png
 
I dont know what do to with them and why you would do either thing? Why would you create a Delta Parquet file in Files (and I would still like it in a folder or a schema and I dont know how to do that.
 
Over just creating a PARQUET File in Files. 
 
And how do I use each one differrently? 
1 ACCEPTED SOLUTION

Hi @DebbieE 
In the context of the current lakehouse the table created in the Files folder is an external table. For understanding the difference between external and internal tables please refer to this link:
https://www.youtube.com/watch?v=09VKDKXqkj4&t=86s

The last statement is true. You won't see external table in semantic model or SQL endpoint.
Hope this helps.

View solution in original post

5 REPLIES 5
v-nikhilan-msft
Community Support
Community Support

Hi @DebbieE 
Thanks for using Fabric Community.
You cannot create folders in Tables. You can only create folders in the Files section. Tables have to be in Tables folder in order to be recognized by other engines like SQL endpoint and semantic model. You can create external tables, i.e. in Files folder but that only be visible for Spark. 
What are you actually trying to achieve? Can you please explain further? This would help me in understanding the issue better.
Thanks

this is an excercise so I can;t tell you what Im trying to do here. i am just trying to understand.  Thats good information so far through. 

 

External tables are confusing me. Are you saying that the PARQUET Table I created in Files is an external table?  That will really help me understand what an external table is if thats the case.

 

"You can create external tables, i.e. in Files folder but that only be visible for Spark. . " Does that mean I cant connect to the PARQUET File in Files with Power BI? I can only do that with the DELTA PARQUET table?

Hi @DebbieE 
In the context of the current lakehouse the table created in the Files folder is an external table. For understanding the difference between external and internal tables please refer to this link:
https://www.youtube.com/watch?v=09VKDKXqkj4&t=86s

The last statement is true. You won't see external table in semantic model or SQL endpoint.
Hope this helps.

Hi @DebbieE 

We haven’t heard from you on the last response and was just checking back to see if your query has resolved. Otherwise, will respond back with the more details and we will try to help.
Thanks

Hi @DebbieE 

We haven’t heard from you on the last response and was just checking back to see if your query has resolved. Otherwise, will respond back with the more details and we will try to help.
Thanks

Helpful resources

Announcements
LearnSurvey

Fabric certifications survey

Certification feedback opportunity for the community.

April Fabric Update Carousel

Fabric Monthly Update - April 2024

Check out the April 2024 Fabric update to learn about new features.

April Fabric Community Update

Fabric Community Update - April 2024

Find out what's new and trending in the Fabric Community.

Top Kudoed Authors