Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
kylechapin
Frequent Visitor

streaming dataset API

Hello,

 

I am trying to create a streaming dataset to receive PUSH notifications from our E-Commerce site.  I can create a webhook and use the URL that is provided when creating a streaming dataset.  

However, I need to put in each and every item line by line.  The JSON that is pushed is very large and would take a long time to input.  Also, the JSON I'm working with has line items within line items... Its not a simple JSON file 

 

I can see the JSON code preview below, but I can't change except by adding another line item.

Is there a way to copy/paste the JSON structure?

8 REPLIES 8
Eric_Zhang
Employee
Employee


@kylechapin wrote:

Hello,

 

I am trying to create a streaming dataset to receive PUSH notifications from our E-Commerce site.  I can create a webhook and use the URL that is provided when creating a streaming dataset.  

However, I need to put in each and every item line by line.  The JSON that is pushed is very large and would take a long time to input.  Also, the JSON I'm working with has line items within line items... Its not a simple JSON file 

 

I can see the JSON code preview below, but I can't change except by adding another line item.

Is there a way to copy/paste the JSON structure?


@kylechapin

I don't quite get your question. Are you trying to push data from one JSON file to Power BI streaming dataset so that you need to compose the Power BI Streaming dataset JSON which is in pretty different JSON structure from your JSON file? If so, your question is actually beyond the topic of this forum. As to how to parse one JSON to a required structure, I'd suggest you post it in the stackoverflow forum. The required JSON struture in Power BI streaming dataset is usually as below, the names and numbers of the fields depend on the specific dataset you created.

[
{
"Product" :"Product1",
"Sales" :98.6,
"Time" :"2017-10-23T08:55:45.961Z"
}
,
{
"Product" :"Product2",
"Sales" :98.6,
"Time" :"2017-10-23T08:55:45.961Z"
}
]

Hi Erik,

 

Thanks for your reply and I'm sorry if I was unclear.  I tried uploading a screenshot.. but couldn't.  

 

Here is an example of the JSON I need to upload:

[
{
"order_number":"12345",
"billing_address": {"first_name":"Bob","address1":"123 Billing Street","phone":"555-555-BILL","city":"Billtown","zip":"K2P0B0"},
"total_order_amount":"29.99"
}
]

 

But I can only seem to get this:

[
{
"order_number":"12345",
"billing_address": "some address",
"total_order_amount":"29.99"
}
]

 

How do I get the part in red to be included in the JSON I set up in Power BI?

 

 


@kylechapin wrote:

Hi Erik,

 

Thanks for your reply and I'm sorry if I was unclear.  I tried uploading a screenshot.. but couldn't.  

 

Here is an example of the JSON I need to upload:

[
{
"order_number":"12345",
"billing_address": {"first_name":"Bob","address1":"123 Billing Street","phone":"555-555-BILL","city":"Billtown","zip":"K2P0B0"},
"total_order_amount":"29.99"
}
]

 

But I can only seem to get this:

[
{
"order_number":"12345",
"billing_address": "some address",
"total_order_amount":"29.99"
}
]

 

How do I get the part in red to be included in the JSON I set up in Power BI?

 

 


@kylechapin

So the column billing_address can be a JSON string as posted? If so, you could try

[
{
"order_number":"12345",
"billing_address":"{\"first_name\":\"Bob\",\"address1\":\"123 Billing Street\",\"phone\":\"555-555-BILL\",\"city\":\"Billtown\",\"zip\":\"K2P0B0\"}",  
"total_order_amount":"29.99"
}
]

Hi Eric,

 

Sorry if I'm being a bug but I'd like to utilize streaming datasets.  This is pretty advanced work, I understand, but I'm willing to go through training myself.  Can you refer me to any kind of detailed reference material?


@kylechapin wrote:

Hi Eric,

 

Sorry if I'm being a bug but I'd like to utilize streaming datasets.  This is pretty advanced work, I understand, but I'm willing to go through training myself.  Can you refer me to any kind of detailed reference material?


@kylechapin

Can you post the specific JSON data in your case? I think it is all about JSON escaping.

By the way, you can push data via Postman to have flexibility when testing.

Hi Eric,

 

The JSON I'm pushing, as I said is large... but here it is:   (I can post a smaller one for a different event if you'd like)

{
	"id": 820982911946154508,
	"email": "jon@doe.ca",
	"closed_at": null,
	"created_at": "2017-11-09T15:40:00-05:00",
	"updated_at": "2017-11-09T15:40:00-05:00",
	"number": 234,
	"note": null,
	"token": "123456abcd",
	"gateway": null,
	"test": true,
	"total_price": "61.00",
	"subtotal_price": "51.00",
	"total_weight": 0,
	"total_tax": "0.00",
	"taxes_included": false,
	"currency": "USD",
	"financial_status": "voided",
	"confirmed": false,
	"total_discounts": "5.00",
	"total_line_items_price": "56.00",
	"cart_token": null,
	"buyer_accepts_marketing": true,
	"name": "#9999",
	"referring_site": null,
	"landing_site": null,
	"cancelled_at": "2017-11-09T15:40:00-05:00",
	"cancel_reason": "customer",
	"total_price_usd": null,
	"checkout_token": null,
	"reference": null,
	"user_id": null,
	"location_id": null,
	"source_identifier": null,
	"source_url": null,
	"processed_at": null,
	"device_id": null,
	"phone": null,
	"customer_locale": "en",
	"app_id": null,
	"browser_ip": null,
	"landing_site_ref": null,
	"order_number": 1234,
	"discount_codes": [],
	"note_attributes": [],
	"payment_gateway_names": ["visa", "bogus"],
	"processing_method": "",
	"checkout_id": null,
	"source_name": "web",
	"fulfillment_status": "pending",
	"tax_lines": [],
	"tags": "",
	"contact_email": "jon@doe.ca",
	"order_status_url": "https:\/\/checkout.shopify.com\/10576560\/orders\/123456abcd\/authenticate?key=abcdefg",
	"line_items": [{
		"id": 866550311766439020,
		"variant_id": null,
		"title": "SUFFERING **bleep** MUG",
		"quantity": 1,
		"price": "26.00",
		"grams": 907,
		"sku": "15091",
		"variant_title": null,
		"vendor": null,
		"fulfillment_service": "manual",
		"product_id": 4129942214,
		"requires_shipping": true,
		"taxable": true,
		"gift_card": false,
		"name": "SUFFERING **bleep** MUG",
		"variant_inventory_management": null,
		"properties": [],
		"product_exists": true,
		"fulfillable_quantity": 1,
		"total_discount": "0.00",
		"fulfillment_status": null,
		"tax_lines": []
	}, {
		"id": 141249953214522974,
		"variant_id": null,
		"title": "TRADER VIC'S LEATHER FLASK",
		"quantity": 1,
		"price": "30.00",
		"grams": 454,
		"sku": "43000",
		"variant_title": null,
		"vendor": null,
		"fulfillment_service": "manual",
		"product_id": 60402303001,
		"requires_shipping": true,
		"taxable": true,
		"gift_card": false,
		"name": "TRADER VIC'S LEATHER FLASK",
		"variant_inventory_management": null,
		"properties": [],
		"product_exists": true,
		"fulfillable_quantity": 1,
		"total_discount": "5.00",
		"fulfillment_status": null,
		"tax_lines": []
	}],
	"shipping_lines": [{
		"id": 271878346596884015,
		"title": "Generic Shipping",
		"price": "10.00",
		"code": null,
		"source": "shopify",
		"phone": null,
		"requested_fulfillment_service_id": null,
		"delivery_category": null,
		"carrier_identifier": null,
		"discounted_price": "10.00",
		"tax_lines": []
	}],
	"billing_address": {
		"first_name": "Bob",
		"address1": "123 Billing Street",
		"phone": "555-555-BILL",
		"city": "Billtown",
		"zip": "K2P0B0",
		"province": "Kentucky",
		"country": "United States",
		"last_name": "Biller",
		"address2": null,
		"company": "My Company",
		"latitude": null,
		"longitude": null,
		"name": "Bob Biller",
		"country_code": "US",
		"province_code": "KY"
	},
	"shipping_address": {
		"first_name": "Steve",
		"address1": "123 Shipping Street",
		"phone": "555-555-SHIP",
		"city": "Shippington",
		"zip": "40003",
		"province": "Kentucky",
		"country": "United States",
		"last_name": "Shipper",
		"address2": null,
		"company": "Shipping Company",
		"latitude": null,
		"longitude": null,
		"name": "Steve Shipper",
		"country_code": "US",
		"province_code": "KY"
	},
	"fulfillments": [],
	"refunds": [],
	"customer": {
		"id": 115310627314723954,
		"email": "john@test.com",
		"accepts_marketing": false,
		"created_at": null,
		"updated_at": null,
		"first_name": "John",
		"last_name": "Smith",
		"orders_count": 0,
		"state": "disabled",
		"total_spent": "0.00",
		"last_order_id": null,
		"note": null,
		"verified_email": true,
		"multipass_identifier": null,
		"tax_exempt": false,
		"phone": null,
		"tags": "",
		"last_order_name": null,
		"default_address": {
			"id": 715243470612851245,
			"customer_id": 115310627314723954,
			"first_name": null,
			"last_name": null,
			"company": null,
			"address1": "123 Elm St.",
			"address2": null,
			"city": "Ottawa",
			"province": "Ontario",
			"country": "Canada",
			"zip": "K2H7A8",
			"phone": "123-123-1234",
			"name": "",
			"province_code": "ON",
			"country_code": "CA",
			"country_name": "Canada",
			"default": false
		}
	}
}

 

Hi Eric,

 

I will give this a try and let you know how it goes.

Thanks!

Hi Eric,

 

I have tried this as you suggested above, in several ways.  I've tried with/wtih out quotes, I've removed the dummy variable values and left the quotes, and removed the quotes

Nothing worked.

Do you have any other suggestions?  Are there other resources I can lean on?

Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors
Top Kudoed Authors