SSIS Tutorial Part 59 | How to Incremental Load in SSIS Using Lookup and Insert & Update

If there are duplicate ID's. Like if I am having a fact table which is having duplicate records

Very good vide Sirji

Very good video. Many thanks.

Please put a video on : How do we add parameter for the ADO source, like using variables to filter the records. select * from table1 where date > ? I am connecting to a DB2 source and retrieving data, it already got 20million records on it, i just what to upload it incrementally on last modified date. need to query the DB2 using the date modified.

tks

Thank you soo much Bhai you saved my day... Less time with more information!! 😊👍

Please reply sir

Which of the following data types power bi does not support and which is a context in power bi

Hi sir please doucts clear sir how does power bi help dig deeper into visual analysis.

I get only o in place off false. How we change

Great way to handle the incremental data insert and update the existing.👍
Just 2-3 days back, I've come accross this scenario where I've used SQL queries (with merge statement) to insert new and update the existing records.
I had 40-50 tables and in each table atleast 30-35 columns. So to manage this I've used SQL queries but your solution is great too. 👍
Nice video keep it up. Keep posting such a great informative videos ❤️

Same table insert update why not possible ?
Stagging table ? Use ?

Thanks for your tutorial. Very good. I have a question, in the step which you use 'OLE DB Command', can that be done also by using 'Execute SQL Task''? Thanks :)

Sir I have a requirement where I need to get data from multiple tables and views and the total records will be around 50000 per day.

The target system uses REST so db data needs to be converted to JSON so how to do this.

scenario - 1 If i want to send data in batch of 5k till it reaches 50k records
scenario-2. Pull 50k records and then convert to json and send it as is(i think its not possible but advise on this approach)

Sir thanks a lot for the video. Lets say a record is deleted from the source table so how do we handle that using lookup we will be either inserting or updating based on EMPID.

keep going bro. It is nice video to understan

Hi Sir, Incremental load is better or CDC is better , when to use both of them , are they same or different?

why do you pulling all 12 records from source every time?

Use a checksum aggregation between each table and compare the checksum. It is quicker. Lookups on SCD are slow if you have n100k or millions of rows.

Sir, Can you please upload one video on SSIS configuration?
Specially XML and SQL Configuration file

Thanks sir for this great video. All doubt gets cleared. Sir, Could you please upload 1 more video on sequence container with example like this?

Great video on incremental load.. Good to see other techniques as well

simple but super!

Great video