copy .csv data to warehouse

[arkiboys]

hello,

using the copy activity in fabric I can take data from .csv file (Which is sitting in azure storage account) and place the data into the table in the warehouse (tblwh) - I am using mapping in the copy activity. all good so far.

Now I have added a few more columns into the tblwh. These new columns are not in the source .csv file.

For example, I have added a new column called presentdate, filepath, fileextension, fullfilepath

Question:

How is it possible I populate these extra columns in the tblwh because the mapping tab in copy activity does not obviously have any source columns for these ?

thanks

[v-gchenna-msft]

Hi @arkiboys ,

Thanks for using Fabric Community.
You can add your additional columns in Copy Activity from here - 

Eg:

After adding additional columns in source can you please click on import schema in mapping?

Eg:

After importing you can run the pipeline

Hope this helps.

[arkiboys]

this is good. thanks

in source copy tab, I use

filename --> $$FILENAME which gives 'some-03-May-2024.csv' --> this is correct

directory_path --> $$FILEPATH but this gives the fullpath such as 'Data/some-new/some-03-May-2024.csv'

whereas I would like to see the directory such as 'Data/some-new'

thanks

[v-gchenna-msft · Accepted solution]

Hi @arkiboys ,

I can also find the same result -

By default FileName and FilePath are designed like that.
As a workaround (assume file path won't change dynamically), you can use custom value -

Hope this is helpful. Please let me know incase of further queries.

[arkiboys]

thank you

[v-gchenna-msft]

Hi @arkiboys ,

Glad to know that your query got resolved. Please continue using Fabric Community on your further queries.

[arkiboys]

hi, I see you have hardcoded the value however my pipeline I would like it to be dynamic as the path will vary since the source varies

any suggestions?

thanks

[v-gchenna-msft]

Hi @arkiboys ,

Can you please share the screenshot of Copy Activity - Source?
I would like to see how are defining file path dynamically? So that I can help you.