Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Grow your Fabric skills and prepare for the DP-600 certification exam by completing the latest Microsoft Fabric challenge.

Reply
Anonymous
Not applicable

Dynamic date data retreival

Hello all,

Im trying to alter the SQL query retreiving all of my data to only retreive Today - 4 days and everything after. This is because im using this specific dataflow to refresh without hitting our SQL server too hard.

the SQL querry is LONG and has countless if statements because powerquery. i wouldnt really know where to insert my statement.
Can anyone help me with this specific instance? 

Context: This query is used in my source table of my dataflow. This dataflow only needs to show very recent data which we want to have "live". I know its not actually going to be live, but thats not the point, we need it refreshed at specific points of the day so my collegues have more to work off. a few days of data will reduce the time allot in compared to the 1400 days its trying to load now, which takes 25 minutes. 

Sql.Database(#"Database Server", #"Database Name", [Query="#(lf)#(lf)select#(lf)#(lf)Leg.""Legnr"", #(lf)Leg.""SalesOrder"",#(lf)ROW_NUMBER() OVER (PARTITION BY Leg.SalesOrder ORDER BY Legnr) AS SalesOrderLine,#(lf)Leg.""LegStatus"", #(lf)Leg.""Planned"",#(lf)Leg.""SalesInvoice"", #(lf)isnull(Leg.EDIProvider,'987654321') as EDIProvider,#(lf)Leg.""Distance"", #(lf)Leg.""OrderLeg"", #(lf)Leg.""PlanGroup"" as 'FK PlanGroup', #(lf)Leg.""FinDate"", #(lf)Leg.""TotalAmount"", #(lf)Leg.""Costs"", #(lf)Leg.""CMRReceived"",#(lf)Leg.[EmailSalesInvoice],#(lf)Leg.[Reference],#(lf)Leg.""CreationUser"",#(lf)Leg.[Cargo],#(lf)--Leg.BeginActivity,#(lf)--leg.EndActivity,#(lf)isnull(B.""Address"",(b.Activitynr + 900000000)) as BeginAddress, #(lf)B.""Date"" as BeginDate, #(lf)B.""DateTill"" as BeginDateTill, #(lf)B.""Time"" as BeginTime, #(lf)B.""TimeTill"" as BeginTimeTill,#(lf)B.""ResourceCombination"" as BeginResourceCombination, #(lf)B.""TripID"" as BeginTripID, #(lf)B.""StartDate"" as BeginStartDate, #(lf)B.""StartTime"" as BeginStartTime, #(lf)B.""EndDate"" as BeginEndDate, #(lf)B.""EndTime"" as BeginEndTime, #(lf)B.""ActivityDuration"" as BeginActivityDuration,#(lf)isnull(E.""Address"",(e.activitynr + 800000000)) as EndAddress, #(lf)E.""Date"" as EndDate, #(lf)E.""DateTill"" as EndDateTill, #(lf)E.""Time"" as EndTime, #(lf)E.""TimeTill"" as EndTimeTill, #(lf)E.""ResourceCombination"" as EndResourceCombination, #(lf)E.""TripID"" as EndTripID, #(lf)E.""StartDate"" as EndStartDate, #(lf)E.""StartTime"" as EndStartTime, #(lf)E.""EndDate"" as EndEndDate, #(lf)E.""EndTime"" as EndEndTime, #(lf)E.""ActivityDuration"" as EndActivityDuration,#(lf)leg.CreationDate,#(lf)Sum(Cargo.""Weight"") as Weight, #(lf)Sum(Cargo.""LoadingMeter"") as LoadingMeter, #(lf)Sum(Cargo.""PalletPlaces"") as PalletPlaces,#(lf)CASE WHEN legnr in (SELECT DISTINCT [leg] AS [pl_orders]#(lf)#(tab)#(tab)#(tab)#(tab)#(tab)FROM SO_Cargo #(lf)#(tab)#(tab)#(tab)#(tab)#(tab)WHERE Unit = 78) THEN 'pl' ELSE NULL END AS [pl order] -- unit 78 = pl#(lf)from SO_Leg as Leg#(lf)left join SO_Activity as B on leg.BeginActivity = B.Activitynr#(lf)left join SO_Activity as E on leg.EndActivity = E.Activitynr#(lf)left join so_cargo as cargo on leg.[legnr] = cargo.Leg#(lf)where FinDate >= DATEADD(year,-3,convert (date,DATEADD(YEAR, DATEDIFF(YEAR,0,GETDATE()) + 1, 0))) #(lf)#(lf)group by #(lf)Leg.""Legnr"", #(lf)Leg.""SalesOrder"",#(lf)Leg.""LegType"", #(lf)Leg.""LegStatus"", #(lf)Leg.""Planned"",#(lf)Leg.""SalesInvoice"", #(lf)Leg.EDIProvider,#(lf)Leg.""Distance"", #(lf)Leg.""OrderLeg"", #(lf)Leg.""PlanGroup"", #(lf)Leg.""FinDate"", #(lf)Leg.""TotalAmount"", #(lf)Leg.""Costs"", #(lf)Leg.""CMRReceived"",#(lf)Leg.[EmailSalesInvoice],#(lf)Leg.[Reference],#(lf)Leg.""CreationUser"",#(lf)Leg.Cargo,#(lf)--Leg.BeginActivity,#(lf)--Leg.EndActivity,#(lf)leg.CreationDate,#(lf)B.Activitynr,#(lf)B.""ActivityKind"", #(lf)B.""Address"", #(lf)B.""Date"", #(lf)B.""DateTill"", #(lf)B.""Time"", #(lf)B.""TimeTill"", #(lf)B.""ResourceCombination"", #(lf)B.""TripID"", #(lf)B.""StartDate"", #(lf)B.""StartTime"", #(lf)B.""EndDate"", #(lf)B.""EndTime"", #(lf)B.""ActivityDuration"",#(lf)E.""ActivityKind"", #(lf)E.""Address"", #(lf)E.""Date"", #(lf)E.""DateTill"", #(lf)E.""Time"", #(lf)E.""TimeTill"",#(lf)E.""ResourceCombination"", #(lf)E.""TripID"", #(lf)E.""StartDate"", #(lf)E.""StartTime"", #(lf)E.""EndDate"", #(lf)E.""EndTime"", #(lf)E.""ActivityDuration"",#(lf)E.Activitynr#(lf)#(lf)#(lf)#(lf)#(lf)"])

 

1 ACCEPTED SOLUTION

Here is your actual SQL statement. You'll need to head over to a SQL forum to get someone to help. Someone here might be able to assist, but a SQL forum as a bunch of people that can assist.

SELECT Leg."legnr", 
       Leg."salesorder", 
       Row_number() 
         OVER ( 
           partition BY Leg.salesorder 
           ORDER BY legnr)                               AS SalesOrderLine, 
       Leg."legstatus", 
       Leg."planned", 
       Leg."salesinvoice", 
       Isnull(Leg.ediprovider, '987654321')              AS EDIProvider, 
       Leg."distance", 
       Leg."orderleg", 
       Leg."plangroup"                                   AS 'FK PlanGroup', 
       Leg."findate", 
       Leg."totalamount", 
       Leg."costs", 
       Leg."cmrreceived", 
       Leg.[emailsalesinvoice], 
       Leg.[reference], 
       Leg."creationuser", 
       Leg.[cargo], 
       --Leg.BeginActivity, 
       --leg.EndActivity, 
       Isnull(B."address", ( b.activitynr + 900000000 )) AS BeginAddress, 
       B."date"                                          AS BeginDate, 
       B."datetill"                                      AS BeginDateTill, 
       B."time"                                          AS BeginTime, 
       B."timetill"                                      AS BeginTimeTill, 
       B."resourcecombination"                           AS 
       BeginResourceCombination, 
       B."tripid"                                        AS BeginTripID, 
       B."startdate"                                     AS BeginStartDate, 
       B."starttime"                                     AS BeginStartTime, 
       B."enddate"                                       AS BeginEndDate, 
       B."endtime"                                       AS BeginEndTime, 
       B."activityduration"                              AS 
       BeginActivityDuration, 
       Isnull(E."address", ( e.activitynr + 800000000 )) AS EndAddress, 
       E."date"                                          AS EndDate, 
       E."datetill"                                      AS EndDateTill, 
       E."time"                                          AS EndTime, 
       E."timetill"                                      AS EndTimeTill, 
       E."resourcecombination"                           AS 
       EndResourceCombination, 
       E."tripid"                                        AS EndTripID, 
       E."startdate"                                     AS EndStartDate, 
       E."starttime"                                     AS EndStartTime, 
       E."enddate"                                       AS EndEndDate, 
       E."endtime"                                       AS EndEndTime, 
       E."activityduration"                              AS EndActivityDuration, 
       leg.creationdate, 
       Sum(Cargo."weight")                               AS Weight, 
       Sum(Cargo."loadingmeter")                         AS LoadingMeter, 
       Sum(Cargo."palletplaces")                         AS PalletPlaces, 
       CASE 
         WHEN legnr IN (SELECT DISTINCT [leg] AS [pl_orders] 
                        FROM   so_cargo 
                        WHERE  unit = 78) THEN 'pl' 
         ELSE NULL 
       END                                               AS [pl order] 
-- unit 78 = pl 
FROM   so_leg AS Leg 
       LEFT JOIN so_activity AS B 
              ON leg.beginactivity = B.activitynr 
       LEFT JOIN so_activity AS E 
              ON leg.endactivity = E.activitynr 
       LEFT JOIN so_cargo AS cargo 
              ON leg.[legnr] = cargo.leg 
WHERE  findate >= Dateadd(year, -3, CONVERT (DATE, Dateadd(year, Datediff(year, 
                                                                 0, 
                                                                 Getdate()) + 1, 
                                                                     0))) 
GROUP  BY Leg."legnr", 
          Leg."salesorder", 
          Leg."legtype", 
          Leg."legstatus", 
          Leg."planned", 
          Leg."salesinvoice", 
          Leg.ediprovider, 
          Leg."distance", 
          Leg."orderleg", 
          Leg."plangroup", 
          Leg."findate", 
          Leg."totalamount", 
          Leg."costs", 
          Leg."cmrreceived", 
          Leg.[emailsalesinvoice], 
          Leg.[reference], 
          Leg."creationuser", 
          Leg.cargo, 
          --Leg.BeginActivity, 
          --Leg.EndActivity, 
          leg.creationdate, 
          B.activitynr, 
          B."activitykind", 
          B."address", 
          B."date", 
          B."datetill", 
          B."time", 
          B."timetill", 
          B."resourcecombination", 
          B."tripid", 
          B."startdate", 
          B."starttime", 
          B."enddate", 
          B."endtime", 
          B."activityduration", 
          E."activitykind", 
          E."address", 
          E."date", 
          E."datetill", 
          E."time", 
          E."timetill", 
          E."resourcecombination", 
          E."tripid", 
          E."startdate", 
          E."starttime", 
          E."enddate", 
          E."endtime", 
          E."activityduration", 
          E.activitynr 

 

Or, post some sample data and tell us what you are trying to do and we can start from scratch with Power Query. 

You should rarely/never use that advanced SQL dialog box. Either create a View on the SQL server, or do it in Power Query. Doing it via an advanced SQL statement can cause permissions issue and prevents simple edits. I could adjust an M statement to only get the last 4 days of info in 2-3 lines that would fold back to the server. Nothing after an advanced SQL statement entry will allow folding.



Did I answer your question? Mark my post as a solution!
Did my answers help arrive at a solution? Give it a kudos by clicking the Thumbs Up!

DAX is for Analysis. Power Query is for Data Modeling


Proud to be a Super User!

MCSA: BI Reporting

View solution in original post

3 REPLIES 3
Greg_Deckler
Super User
Super User

@Anonymous - It looks like you are using a SQL Statement in your query. This is done in the Advanced portion of the Source step. I think that you would want to paste this SQL code instead and modify it to only get the last 4 days.


Follow on LinkedIn
@ me in replies or I'll lose your thread!!!
Instead of a Kudo, please vote for this idea
Become an expert!: Enterprise DNA
External Tools: MSHGQM
YouTube Channel!: Microsoft Hates Greg
Latest book!:
The Definitive Guide to Power Query (M)

DAX is easy, CALCULATE makes DAX hard...
Anonymous
Not applicable


@Greg_Deckler wrote:

@Anonymous - It looks like you are using a SQL Statement in your query. This is done in the Advanced portion of the Source step. I think that you would want to paste this SQL code instead and modify it to only get the last 4 days.


This was done by a  previous engineer, ive been looking for a while now, is there a way to get this query up again? 

Here is your actual SQL statement. You'll need to head over to a SQL forum to get someone to help. Someone here might be able to assist, but a SQL forum as a bunch of people that can assist.

SELECT Leg."legnr", 
       Leg."salesorder", 
       Row_number() 
         OVER ( 
           partition BY Leg.salesorder 
           ORDER BY legnr)                               AS SalesOrderLine, 
       Leg."legstatus", 
       Leg."planned", 
       Leg."salesinvoice", 
       Isnull(Leg.ediprovider, '987654321')              AS EDIProvider, 
       Leg."distance", 
       Leg."orderleg", 
       Leg."plangroup"                                   AS 'FK PlanGroup', 
       Leg."findate", 
       Leg."totalamount", 
       Leg."costs", 
       Leg."cmrreceived", 
       Leg.[emailsalesinvoice], 
       Leg.[reference], 
       Leg."creationuser", 
       Leg.[cargo], 
       --Leg.BeginActivity, 
       --leg.EndActivity, 
       Isnull(B."address", ( b.activitynr + 900000000 )) AS BeginAddress, 
       B."date"                                          AS BeginDate, 
       B."datetill"                                      AS BeginDateTill, 
       B."time"                                          AS BeginTime, 
       B."timetill"                                      AS BeginTimeTill, 
       B."resourcecombination"                           AS 
       BeginResourceCombination, 
       B."tripid"                                        AS BeginTripID, 
       B."startdate"                                     AS BeginStartDate, 
       B."starttime"                                     AS BeginStartTime, 
       B."enddate"                                       AS BeginEndDate, 
       B."endtime"                                       AS BeginEndTime, 
       B."activityduration"                              AS 
       BeginActivityDuration, 
       Isnull(E."address", ( e.activitynr + 800000000 )) AS EndAddress, 
       E."date"                                          AS EndDate, 
       E."datetill"                                      AS EndDateTill, 
       E."time"                                          AS EndTime, 
       E."timetill"                                      AS EndTimeTill, 
       E."resourcecombination"                           AS 
       EndResourceCombination, 
       E."tripid"                                        AS EndTripID, 
       E."startdate"                                     AS EndStartDate, 
       E."starttime"                                     AS EndStartTime, 
       E."enddate"                                       AS EndEndDate, 
       E."endtime"                                       AS EndEndTime, 
       E."activityduration"                              AS EndActivityDuration, 
       leg.creationdate, 
       Sum(Cargo."weight")                               AS Weight, 
       Sum(Cargo."loadingmeter")                         AS LoadingMeter, 
       Sum(Cargo."palletplaces")                         AS PalletPlaces, 
       CASE 
         WHEN legnr IN (SELECT DISTINCT [leg] AS [pl_orders] 
                        FROM   so_cargo 
                        WHERE  unit = 78) THEN 'pl' 
         ELSE NULL 
       END                                               AS [pl order] 
-- unit 78 = pl 
FROM   so_leg AS Leg 
       LEFT JOIN so_activity AS B 
              ON leg.beginactivity = B.activitynr 
       LEFT JOIN so_activity AS E 
              ON leg.endactivity = E.activitynr 
       LEFT JOIN so_cargo AS cargo 
              ON leg.[legnr] = cargo.leg 
WHERE  findate >= Dateadd(year, -3, CONVERT (DATE, Dateadd(year, Datediff(year, 
                                                                 0, 
                                                                 Getdate()) + 1, 
                                                                     0))) 
GROUP  BY Leg."legnr", 
          Leg."salesorder", 
          Leg."legtype", 
          Leg."legstatus", 
          Leg."planned", 
          Leg."salesinvoice", 
          Leg.ediprovider, 
          Leg."distance", 
          Leg."orderleg", 
          Leg."plangroup", 
          Leg."findate", 
          Leg."totalamount", 
          Leg."costs", 
          Leg."cmrreceived", 
          Leg.[emailsalesinvoice], 
          Leg.[reference], 
          Leg."creationuser", 
          Leg.cargo, 
          --Leg.BeginActivity, 
          --Leg.EndActivity, 
          leg.creationdate, 
          B.activitynr, 
          B."activitykind", 
          B."address", 
          B."date", 
          B."datetill", 
          B."time", 
          B."timetill", 
          B."resourcecombination", 
          B."tripid", 
          B."startdate", 
          B."starttime", 
          B."enddate", 
          B."endtime", 
          B."activityduration", 
          E."activitykind", 
          E."address", 
          E."date", 
          E."datetill", 
          E."time", 
          E."timetill", 
          E."resourcecombination", 
          E."tripid", 
          E."startdate", 
          E."starttime", 
          E."enddate", 
          E."endtime", 
          E."activityduration", 
          E.activitynr 

 

Or, post some sample data and tell us what you are trying to do and we can start from scratch with Power Query. 

You should rarely/never use that advanced SQL dialog box. Either create a View on the SQL server, or do it in Power Query. Doing it via an advanced SQL statement can cause permissions issue and prevents simple edits. I could adjust an M statement to only get the last 4 days of info in 2-3 lines that would fold back to the server. Nothing after an advanced SQL statement entry will allow folding.



Did I answer your question? Mark my post as a solution!
Did my answers help arrive at a solution? Give it a kudos by clicking the Thumbs Up!

DAX is for Analysis. Power Query is for Data Modeling


Proud to be a Super User!

MCSA: BI Reporting

Helpful resources

Announcements
RTI Forums Carousel3

New forum boards available in Real-Time Intelligence.

Ask questions in Eventhouse and KQL, Eventstream, and Reflex.

MayPowerBICarousel1

Power BI Monthly Update - May 2024

Check out the May 2024 Power BI update to learn about new features.

Top Solution Authors
Top Kudoed Authors