cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
sp_mike
Helper II
Helper II

Python add data to existing table based on existing data

I have data imported in power bi. I want to use one of the columns as input to fetch data from the internet. The data I receive I want to add to anonther (existing table). Something like this:

 

import pandas as pd

df = dataset #Required step in Power BI

gb_list = df['Column2'].T.tolist()
gb_list = list(dict.fromkeys(gb_list))

---Get data from internet ----

---Write result to other table----

df2 = second table
second table.column1 = bla, 
second table.colum2 = blabla
etc
etc

 

 

Can this be done, using Python?

1 ACCEPTED SOLUTION

Hi, @sp_mike 

 

Could you consider turning off both options in power query?

vjaneygmsft_0-1643700017424.png

And try not to add data sources that you don't want to refresh after publishing to the service.

 

Did I answer your question? Please mark my reply as solution. Thank you very much.
If not, please feel free to ask me.

Best Regards,
Community Support Team _ Janey

View solution in original post

7 REPLIES 7
sp_mike
Helper II
Helper II

Ok, indeed. Thanks.

However, I use data from an already existing dataset as variable for my second web query. How can I fetch the data from this first dataset. Sy this would be a table called "test"

 

I would like to do something like : 

df = test # my already existing table
gb_list = df['Column2'].T.tolist()

 

How can I do that?

AFAIK, you can't use py to read datasets existing in PBI because they are not stored as separate files.

 

PQ itself is a powerful ETL tool; I don't see the point resorting to a third-party script to manipulate dataset already existing within PQ.

gb_list = test['Column2']

Thanks to the great efforts by MS engineers to simplify syntax of DAX! Most beginners are SUCCESSFULLY MISLED to think that they could easily master DAX; but it turns out that the intricacy of the most frequently used RANKX() is beyond their comprehension!

DAX is simple, but NOT EASY!

What I try to do is scrape a webpage with a lot of data and each time I start PBI I want to add the latest records and not load the everything again because this would take ages. Besides the initial data comes from one webpage and the added data from another. I hope this makes sense.

Hi, @sp_mike 

 

Could you consider turning off both options in power query?

vjaneygmsft_0-1643700017424.png

And try not to add data sources that you don't want to refresh after publishing to the service.

 

Did I answer your question? Please mark my reply as solution. Thank you very much.
If not, please feel free to ask me.

Best Regards,
Community Support Team _ Janey

It was getting a bit complicated to collect all data directly in powerBI. For now I collect the main part in mySQL (using python scipt) and use mySQL as source in PowerBI

Thanks, I was not aware of these settings.

CNENFRNL
Super User
Super User

CNENFRNL_0-1643213201991.png

 


Thanks to the great efforts by MS engineers to simplify syntax of DAX! Most beginners are SUCCESSFULLY MISLED to think that they could easily master DAX; but it turns out that the intricacy of the most frequently used RANKX() is beyond their comprehension!

DAX is simple, but NOT EASY!

Helpful resources

Announcements
Microsoft Build 768x460.png

Microsoft Build is May 24-26. Have you registered yet?

Come together to explore latest innovations in code and application development—and gain insights from experts from around the world.

charticulator_carousel_with_text (1).png

Charticulator Design Challenge

Put your data visualization and design skills to the test! This exciting challenge is happening now through May 31st!

Power BI Dev Camp Session 22 768x460.jpg

Check it out!

Mark your calendars and join us on Thursday, May 26 at 11a PDT for a great session with Ted Pattison!