Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
Anonymous
Not applicable

reading data from aws s3

Can power bi read data from aws s3 ( json or any other format ) ?

1 ACCEPTED SOLUTION
PhilipTreacy
Super User
Super User

Hi @Anonymous 

S3 has security that can be applied that can restrict access to a resource, be that an individual file or the entire bucket.  In S3 they refer to a bucket - this is the container for your data. You can think of it as a folder.

The security on this bucket can be set such that it is open to everyone, just like a website. 

Websites allow people to read (see) the files in that site, if they didn't the site wouldn't work because your browser couldn't read the site contents.

So, if your S3 security is set so that the bucket is open, you can use PQ to access the files in that bucket.

If your S3 securty is more restrictive then you will need to use the S3 API as you will need to provide authentication (username/password) to login and access the bucket resources.

If your want to connect to S3 to read meta-data, that is things like reports on usage, then you will need to go through the API too.

Regards

Phil



Did I answer your question? Then please mark my post as the solution.
If I helped you, click on the Thumbs Up to give Kudos.


Blog :: YouTube Channel :: Connect on Linkedin


Proud to be a Super User!


View solution in original post

3 REPLIES 3
PhilipTreacy
Super User
Super User

Hi @Anonymous 

S3 has security that can be applied that can restrict access to a resource, be that an individual file or the entire bucket.  In S3 they refer to a bucket - this is the container for your data. You can think of it as a folder.

The security on this bucket can be set such that it is open to everyone, just like a website. 

Websites allow people to read (see) the files in that site, if they didn't the site wouldn't work because your browser couldn't read the site contents.

So, if your S3 security is set so that the bucket is open, you can use PQ to access the files in that bucket.

If your S3 securty is more restrictive then you will need to use the S3 API as you will need to provide authentication (username/password) to login and access the bucket resources.

If your want to connect to S3 to read meta-data, that is things like reports on usage, then you will need to go through the API too.

Regards

Phil



Did I answer your question? Then please mark my post as the solution.
If I helped you, click on the Thumbs Up to give Kudos.


Blog :: YouTube Channel :: Connect on Linkedin


Proud to be a Super User!


PhilipTreacy
Super User
Super User

Hi @Anonymous 

PBI has a built in JSON connector so yes it can read/receive JSON data.  And text, or XML, or lots of other formats.

Regarding connecting to S3 in general, do you mean to read data from a bucket or to read data via their API?

If the bucket is configured appropriately then you can read data/files from it like any other web site.

If you need to use the API then you can also do that.  You just need to configure your web connector to connect in the right way.

More info here

Amazon S3 REST API Introduction - Amazon Simple Storage Service

https://www.myonlinetraininghub.com/getting-started-with-apis-in-power-query

https://www.myonlinetraininghub.com/connecting-to-an-oauth-api-like-paypal-with-power-query

Regards

Phil



Did I answer your question? Then please mark my post as the solution.
If I helped you, click on the Thumbs Up to give Kudos.


Blog :: YouTube Channel :: Connect on Linkedin


Proud to be a Super User!


Anonymous
Not applicable

Thank you, just starting out so not sure how my app team will  populate in s3. 

 

If the bucket is configured appropriately then you can read data/files from it like any other web site. - what does this mean ? what type of configuration is required ?

Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.