08 Incremental data load in SSIS based on lastupdated or modifieddate | SSIS real time scenarios

Sir, your ssis playlist enough to land a job as a ssis developer

Salam aleikom Sorry if I insist, this query hi I need to perform this in SISS how can I perform two aggragate? because once I need to group by month and then I need a new column which is grouped by year, PLEASE HELP ME


SELECT
d.month, d.year,
SUM(c.crime_gravity) AS MonthlyGravity,
SUM(SUM(c.crime_gravity)) OVER (PARTITION BY d.year) AS AnnualTotalGravity,
(SUM(c.crime_gravity) * 100.0) / NULLIF(SUM(SUM(c.crime_gravity)) OVER (PARTITION BY d.year), 0) AS PercentageGravity
FROM
Date d
JOIN
Custody c ON c.date_fk = d.date_pk
GROUP BY
d.month, d.year
ORDER BY d.month, d.year;
Has to be performed in SISS without usino sql just the tools inside SISS like lookup aggregation ect.
Please help me

hi I need to perform this in SISS how can I perform two aggragate? because once I need to group by month and then I need a new column which is grouped by year, PLEASE HELP ME


SELECT
d.month, d.year,
SUM(c.crime_gravity) AS MonthlyGravity,
SUM(SUM(c.crime_gravity)) OVER (PARTITION BY d.year) AS AnnualTotalGravity,
(SUM(c.crime_gravity) * 100.0) / NULLIF(SUM(SUM(c.crime_gravity)) OVER (PARTITION BY d.year), 0) AS PercentageGravity
FROM
Date d
JOIN
Custody c ON c.date_fk = d.date_pk
GROUP BY
d.month, d.year
ORDER BY d.month, d.year;

Hello Ahmed,

I hope you are doing well,

how do validate the foreign key in fact table already exists in the dim table before loading the data?
should I use two lookup components? One for left join and the second one for split data to new records and updated records?

thanks,

Hi Aqil
I'm following your SSIS playlist. It's really helpful.
I have to load the data from oracle to SQL server. I do no have timestamp in oracle DB but i have date of transaction. Please create a video on this topic else if you have any article pls share the link . It's urgent . It would be great if you help me .🙂🙂🙂🙂

Good expalnation. Very helpful video @learnssis.
Can you please share the dataset fo this video?
Can you also create a video which has Incremental load (with Creation Date)+ Staging Layer + SCD type 1 & 2 + Audit table for ETL load date all in one SSIS package ?
Thanks

Hi Ahmed,

Thanks for the great video.

I got this error after the execution

Execute SQL Task Error: Executing the query
declare @updatedRecords int

UPDATE a
SET a...." failed with the following error: "Multiple-step OLE DB operation generated errors.
Check each OLE DB status value, if available. No work was done.". Possible failure reasons:
Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.

Hello bro, I did the work step by step, but the result was 10 records before the update. Please advise🤨

Please put a video on : How do we add parameter for the ADO source, like using variables to filter the records. select * from table1 where date > ? I am connecting to a DB2 source and retrieving data, it already got 20million records on it, i just what to upload it incrementally on last modified date. need to query the DB2 using the date modified.

Nice

Hi, Sir your lecture is excellent it is exactly what I was looking for in my project but 2 things I want to notify here first, please try not to use the same named Databases or tables for the source and destination it is making too much confusion and complicated to understand and second, the provided script is not sufficient as it is a 1-hour session and you just provided a single statement please check. I also want to thank you for your efforts in doing these sessions and helping people like me. God Bless you. Keep it up!!

Hi bro, i want learn SSIS from u pls how to contact and pls. Msg me

Hi Ahmed,

Can you please let me know. How to implement for multiple tables..where in my environment there are 300 tables in the database will be modified/inserted every day.

Thanks.. this video helped a lot..
And I have implemented this for one table.

Hello Ahmed,

I pray to God to bless you.
Extremely useful video.
Can you please make a video illustrating how to handle the incremental load in the dimensions table?
for example, what happens if the value in the dimension table changes, do we need to change the value in all fact table or, just change it based on the time.

Best regards,
Alnebras.

The video is cool. But how can you make one for a fact table?

Excellent video as usual, just a question, is this the best way to load large number of (60 million or more) rows in incremental loading or there is another way?

Question:Is it possible in Loading from Different Layout tables Dynamically using Foreach Loop ? Never saw any example for that over the net. Please can you do that for us, Mr Aqil

Thanks man, you're good at it!

Thank you very much!

How to do lookup tarnsformation from SQL server to oracle sql

hi nice explanation thank you, can you show the same looping through list of tables in a config table, with dynamic columns mapping and updating/ insert using above concept

Would you please make a video on incremental load from oracle to SQL server

Hi Aquil, could u make video on incremental load data through MSSQL Server to Sql Server. Thanks in advance.

could u make video on incremental load data through stored Procedure.Thanks in advance.

can we do the same when the source and destination on different servers?

This is the best way of doing incremental load

Thank you for this great tutorial. would you please share all the codes used in this tutorial?

Thank you, this was a great tutorial.

nice content but very long video

Tx for the detailed explanation

Great tutorials, thanks.
How to implement scd type 2 dimension in SSIS without using SCD component?

Is there any way to update the destination table without creating a ZZ_EmailAddress temporary table?

Hi sir I know your email ? to discuss

Hi Aqil,I have received an Excel file weekly and I need to load the data into Salesforce destination with incremental load using SQL procedure. Can you make video for this scenario.

Aqil, can you do video's on real time scenarios and best practices

Bro need a help from you..... preparing for ssis interview need your suggestions if possible can u provide mail or contact number?

Can you please share dataset

Hi Aqil,
Very very thanks lot