Load Data from GCS to BigQuery using Dataflow

Nice video ..I am able to execute the DataFlow.. Thanks

Hi,
I have been using same approach like you but with different CSV file(UDF is same) but I am getting following error (Loyalty Number is Integer column):

Error message from worker: org.apache.beam.sdk.util.UserCodeException: java.util.concurrent.CompletionException: javax.script.ScriptException: <eval>:5:12 Expected ; but found Number
obj.Loyalty Number = values[0];
^ in <eval> at line number 5 at column number 12

Can you tell me what the error is actually?

I need to remove the header rows as this is getting populated. How to do that?

can you pls share .csv file

Hello I am getting the error below.

org.apache.beam.sdk.util.UserCodeException: java.lang.RuntimeException: Error parsing schema gs://fazendo/mentloja.json

Caused by: java.lang.RuntimeException
Caused by: org.json.JSONException

Can you help me?

Can you send me that CSV.format file and all three files to my mail id..??

One more Question:- Why do we need to specify temp folders here?.

Can you also attach the .csv file so that we can download and use?

Hello I am getting below error.
org.apache.beam.sdk
.util.UserCodeException: java.lang.NoSuchMethodException: No such function transform

at org.apache.beam.sdk.util.UserCodeException.wrap(UserCodeException.java:39)

why so

How to load csv file with comma in data? do you know how to escape the comma? thanks

How can we load the same data from csv file to pubsub topic and then through dataflow job in bigquey ?

How to create USD as I do not have any java knowledge