Combining multiple datasets - Data Analysis with Python and Pandas p.5

Great intro to pandas, thanks a lot! Now, I have a little problem, if anyone have some advice what to look after: At the merge step I get an error that at some point says: KeyError: 'County'. I looked in the structure of both dataframes and they have this column...and all until this point is working well. I googled and got some discussions but not something to fix it. Thanks

Will you also be showing how to train ConvLSTM2D models? I have such a need and haven't come across good instructions yet.

Hi Harrison, thank you so much for your tutorials. I noticed that there is someone from Istanbul plagiarizing your content. I don't know if you already know that and if it bothers you but they LITERALLY copied your course ML tutorial with Python, I think you can find them easily though google search if you wish to claim your rights. All the best, gonna proceed watching your videos now :)

woah, in the beginning, he did not import numpy and it still worked. hehhh????

well, after the numpy incident, I'm starting to believe that sentdex is a god from above, I mean, he gives us riches(information about programming), he helps us with our problems, and, he seems to have some mythical power over technology.

I'm convinced

Dude, ur my digital master! Much learning, i do.

eh! another mug!

This is certainly very helpful! Can you make a video on efficient big dataset process, like 100M plus?

really sir your video is great, it help me a lot in my studies

@sentdex Instead of:
act_min_wage=act_min_wage.replace(0,np.NaN).dropna(axis=1)
Try:
act_min_wage=act_min_wage.loc[:,(act_min_wage.mean()!=0)]
Then you'll have data for get_min_wage(2015,'Texas'), get_min_wage(2015,'Flordia')!... and bunch of other states you zeroed for the year 2015!

Fantastic topic, I was actually interested to see the results as well as to learn about pandas

Muad'Dib no longer needs the weirding module I see... Pandas works for him without import!


Soon he will call the great worm and lead us to the spice.

For some reason I kept getting:


KeyError: 'County'


from:


for df in [county_2015, pres16]:
df.set_index(["County", "State"], inplace = True)


Couldn't figure out why... but a great video, as usual.

If you have merge problem use 'right_index = True', 'left_index = True' instead of 'on'. And since pandas data type 'Object' indicates mixed types, you need to convert pres16['County'] to string.
One way of doing this: "pres16['County'] = pres16['County'].astype(str)" before you change "State" and "County" to indices.

The reason Mississippi has a bunch of NaNs is because the min_wage data does not have Mississippi as an index, they left that state out for some reason. min_wage.columns.unique() you will find Mississippi does not exist.

At what point of time is that state_abbv df created and saved. I can't find it anywhere. Also from where is it saved? Niether i am able to create such df using the given data nor i am able to find it anywhere.

IMPORT!!!

the donut cup has always been one of my favorites

what: inplace = true stands for ?

I love your sense of humor. You really love what you are doing and really appreciate your efforts mate! Thanks

FileNotFoundError: File b'datasets/state_abbv.csv' does not exist


I am getting this error. Please can anyone help me out.

You did a great job Harrison I am having some trouble following along though I have gone through your basics a few times and have learnt them from other places too, any tips?

if anyone recently had trouble importing the second dataset, try encoding the file:
df = pd.read_csv("us-minimum-wage-by-state-from-1968-to-2017\Minimum Wage Data.csv", encoding= 'unicode_escape')

I get a: "KeyError: 'County'", in the for loop.

Someone knows how to fix it?

Would have made sense to replace the Rate with the average rate, grouped by State, then keep the unique values for State and then do the mapping of the new column? to speed up things

cool coffee cup... i am smiling watching this coz it is gonna save my day!

Your tutorials are really fantastic!!
and you have a wonderful mug collection

IMPORT

Thank you!

My  M1 MBP took 33.2 seconds!

Having an issue at the very beggining with the ["Low.2018"] index

There's no column named like that on the dataset, nothing found to rename
"None of [Index(["Low.2018"], dtype='object')] are in the [columns]"

Can you help?

Miss him my pres

As of today, the "result" file is a json. I can't seem to find a way to convert it to a csv. Do you have any idea? did anyone have the same problem?

To merge unemp_county and get_min_wage, I think the fatest method would be to use pd.merge.
The key idea is to first convert get_min_wage back to the long format with "Year" and "State" as (multi-)indices, then we can directly use them as the merge key (by setting right_index=True in pd.merge).


Something like

pd.merge(
unemp_county,
get_min_wage.stack().to_frame(),
how='left',
left_on=["Year", "State"],
right_index=True
)

would work.

i dont understand why you said we'll merge votes with unemployment rate then you merged it with minimum wage???? where did you get the set from