Tensorflow Input Pipeline | tf Dataset | Deep Learning Tutorial 44 (Tensorflow, Keras & Python)
Code: https://github.com/codebasics/deep-learning-keras-tf-tutorial/blob/master/44_tf_data_pipeline/tf_data_pipeline.ipynb
Exercise: https://github.com/codebasics/deep-learning-keras-tf-tutorial/blob/master/44_tf_data_pipeline/Exercise/tf_data_pipeline_exercise.md
Stackoverflow article: https://stackoverflow.com/questions/53514495/what-does-batch-repeat-and-shuffle-do-with-tensorflow-dataset
⭐️ Timestamps ⭐️
00:00 Introduction
00:21 Theory
07:58 Coding
31:34 Exercise
Do you want to learn technology from me? Check https://codebasics.io/ for my affordable video courses.
Deep learning playlist: https://www.youtube.com/playlist?list=PLeo1K3hjS3uu7CxAacxVndI4bE_o3BDtO
Machine learning playlist : https://www.youtube.com/playlist?list=PLeo1K3hjS3uvCeTYTeyfe0-rN5r8zn9rw
🔖Hashtags🔖
#tfdatapipeline #tfdatashuffle #tfdataapi #tfdataexample #tfdatadatasettutorial #tensorflowinputpipeline #deeplearningtutorial #tensorflowtutorial
🌎 Website: https://codebasics.io/
🎥 Codebasics Hindi channel: https://www.youtube.com/channel/UCTmFBhuhMibVoSfYom1uXEg
#️⃣ Social Media #️⃣
🔗 Discord: https://discord.gg/r42Kbuk
📸 Instagram: https://www.instagram.com/codebasicshub/
🔊 Facebook: https://www.facebook.com/codebasicshub
📱 Twitter: https://twitter.com/codebasicshub
📝 Linkedin (Personal): https://www.linkedin.com/in/dhavalsays/
📝 Linkedin (Codebasics): https://www.linkedin.com/company/codebasics/
❗❗ DISCLAIMER: All opinions expressed in this video are of my own and not that of my employers'.
Тэги:
#tf_data_pipeline #tf_data_shuffle #tf_data_api #tf.data.dataset_tutorial #tensorflow_input_pipeline #tf_input_pipeline #tensorflow_data_shuffle #tensorflow_data_api #input_pipeline_tensorflow #input_pipeline #tensorflow_pipeline #tensorflow_pipeline_example #tensorflow_dataset #tensorflow_input_pipeline_tutorial #tensorflow_data_pipeline #tensorflow_input_pipeline_performance #Input_data_pipeline #tf.data_pipeline #tensorflow_datasets #loading_data_tensorflowКомментарии:
does this input pipeline also applicable for hyperspectral images?
Ответить
tf_dataset = tf_dataset.filter(lambda x: x>0)
for sales in tf_dataset.np():
print(sales)
AttributeError Traceback (most recent call last)
<ipython-input-7-6d7e945f4009> in <module>
1 tf_dataset = tf_dataset.filter(lambda x: x>0)
----> 2 for sales in tf_dataset.np():
3 print(sales)
AttributeError: 'FilterDataset' object has no attribute 'np'
i love you man. Been struggling with tf for 2 months as I only have experience with pandas. The theory part was so helpful in understanding why tf is the way it is. And obv the coding part too. Thank you so much!
Ответить
i wish to learn on both deep learning and python through you.
Ответить
Excellent tutorial! Thank you
Ответить
Thanks for great explanation! I've got two questions.
1. You said that it loads data in batches from disk how does shuffling work? Data are sampled from multiple source data then made into one batch or somehow all data is shuffled from disk?
2. I am trying to write tfrecords from pandas dataframe, how to split x,y within tf.data.dataset so it can be trained? After reading tfrecords I have dictionary of features(tensors).
What if folders are not clearly separated as cats and dogs.. and we have just one folder of all images of cats and dogs.
Ответить
if anyone gets this error: `InvalidArgumentError: Unknown image file format. One of JPEG, PNG, GIF, BMP required.`
just delete file `Best Dog & Puppy Health Insurance Plans....jpg` in dogs folder.
This was crazy useful!
Ответить
You are helping the data science community in an excellent way. keep going on and all the power to you. Thanks! and a very small token of appreciation
Ответить
This is awesome!!!!
Ответить
Great Explanation
Ответить
Enjayable presentation. But I have 64GB on MY laptop. :P
Ответить
If I get some image matrix data and save it as a dataframe, how do I pass it into the dataset as a feature? The from_tensor_slices method will report "ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type numpy.ndarray)." Thanks everyone for your help!
Ответить
Very nice tutorial. I wonder how to generate dataset with random numbers, for example vector with uniform distribution in range <0,1> with defined size to use while fiting with defined number of epochs and defined batch size. Is possible to use for this purpose tf.data.experimental.RandomDataset in tf 2.10 ?
Ответить
Man, this really helped me out. I was overcomplicating things. Thanks a bunch!
Ответить
Hii, thanks a lot for the video , very useful, can you please upload tutorial on creating a custom dataset from parallel corpus of data for training ? unable to figure out
Ответить
