机器学习 / 深度学习 / 自然语言处理
CNN classifier for the MNIST dataset
Instructions ¶
How to submit ¶, the mnist dataset ¶, load and preprocess the data ¶, build the convolutional neural network model ¶, compile the model ¶, fit the model to the training data ¶, plot the learning curves ¶, evaluate the model ¶, model predictions ¶.
<!DOCTYPE html>
Programming Assignment ¶
Cnn classifier for the mnist dataset ¶.
In this notebook, you will write code to build, compile and fit a convolutional neural network (CNN) model to the MNIST dataset of images of handwritten digits.
Some code cells are provided you in the notebook. You should avoid editing provided code, and make sure to execute the cells in order to avoid unexpected errors. Some cells begin with the line:
#### GRADED CELL ####
Don't move or edit this first line - this is what the automatic grader looks for to recognise graded cells. These cells require you to write your own code to complete them, and are automatically graded when you submit the notebook. Don't edit the function name or signature provided in these cells, otherwise the automatic grader might not function properly. Inside these graded cells, you can use any functions or classes that are imported below, but make sure you don't use any variables that are outside the scope of the function.
Complete all the tasks you are asked for in the worksheet. When you have finished and are happy with your code, press the Submit Assignment button at the top of this notebook.
Let's get started! ¶
We'll start running some imports, and loading the dataset. Do not edit the existing imports in the following cell. If you would like to make further Tensorflow imports, you should add them here.
In this assignment, you will use the MNIST dataset . It consists of a training set of 60,000 handwritten digits with corresponding labels, and a test set of 10,000 images. The images have been normalised and centred. The dataset is frequently used in machine learning research, and has become a standard benchmark for image classification models.
- Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. "Gradient-based learning applied to document recognition." Proceedings of the IEEE, 86(11):2278-2324, November 1998.
Your goal is to construct a neural network that classifies images of handwritten digits into one of 10 classes.
First, preprocess the data by scaling the training and test images so their values lie in the range from 0 to 1.
We are now ready to construct a model to fit to the data. Using the Sequential API, build your CNN model according to the following spec:
- The model should use the input_shape in the function argument to set the input size in the first layer.
- A 2D convolutional layer with a 3x3 kernel and 8 filters. Use 'SAME' zero padding and ReLU activation functions. Make sure to provide the input_shape keyword argument in this first layer.
- A max pooling layer, with a 2x2 window, and default strides.
- A flatten layer, which unrolls the input into a one-dimensional tensor.
- Two dense hidden layers, each with 64 units and ReLU activation functions.
- A dense output layer with 10 units and the softmax activation function.
In particular, your neural network should have six layers.
You should now compile the model using the compile method. To do so, you need to specify an optimizer, a loss function and a metric to judge the performance of your model.
Now you should train the model on the MNIST dataset, using the model's fit method. Set the training to run for 5 epochs, and return the training history to be used for plotting the learning curves.
We will now plot two graphs:
- Epoch vs accuracy
- Epoch vs loss
We will load the model history into a pandas DataFrame and use the plot method to output the required graphs.
Finally, you should evaluate the performance of your model on the test set, by calling the model's evaluate method.
Let's see some model predictions! We will randomly select four images from the test data, and display the image and label for each.
For each test image, model's prediction (the label with maximum probability) is shown, together with a plot showing the model's categorical distribution.
Congratulations for completing this programming assignment! In the next week of the course we will take a look at including validation and regularisation in our model training, and introduce Keras callbacks.
- 上一篇： Adding weight initialisers
- 下一篇： Optimizer, loss functions, metrics
MNIST - Convolutions
First, we load the data using MLDatasets.jl :
The covariate data ( x ) were named 3 as these are three-dimensional arrays, containing the height x width x number of images. The training data are vectors indicating the digit.
SimpleChains' convolutional layers expect that we have a channels-in dimension, so we shape the images to be four dimensional It also currently defaults to 1-based indexing for its categories, so we shift all categories by 1.
We now define our model, LeNet5:
We define the inputs as being statically sized (28,28,1) images. Specifying the input sizes allows these to be checked. Making them static, which we can do either in our simple chain, or by adding static sizing to the images themselves using a package like StrideArrays.jl or HybridArrays.jl . These packages are recommended for allowing you to mix dynamic and static sizes; the batch size should probably be left dynamic, as you're unlikely to want to specialize code generation on this, given that it is likely to vary, increasing compile times while being unlikely to improve runtimes.
In SimpleChains , the parameters are not a part of the model, but live as a separate vector that you can pass around to optimizers of your choosing. If you specified the input size, you create a random initial parameter vector corresponding to the model:
The convolutional layers are initialized with a Glorot (Xavier) uniform distribution, while the dense layers are initialized with a Glorot (Xaviar) normal distribution. Biases are initialized to zero. Because the number of parameters can be a function of the input size, these must be provided if you didn't specify input dimension. For example:
To allow training to use multiple threads, you can create a gradient matrix, with a number of rows equal to the length of the parameter vector p , and one column per thread. For example:
Here, we're estimating that the number of physical cores is half the number of threads on an x86_64 system, which is true for most – but not all!!! – of them. Otherwise, we're assuming it is equal to the number of threads. This is of course also likely to be wrong, e.g. recent Power CPUs may have 4 or even 8 threads per core. You may wish to change this, or use Hwloc.jl for an accurate number.
Now that this is all said and done, we can train for 10 epochs using the ADAM optimizer with a learning rate of 3e-4 , and then assess the accuracy and loss of both the training and test data:
Training for an extra 10 epochs should be fast on most systems. Performance is currently known to be poor on the M1 (PRs welcome, otherwise we'll look into this eventually), but should be good/great on systems with AVX2/AVX512:
Theme documenter-light documenter-dark
This document was generated with Documenter.jl version 0.27.24 on Tuesday 28 February 2023 . Using Julia version 1.8.5.
Grading Error with Programming Assignment: Improve MNIST with convolutions
I keep getting the error shown below for Programming Assignment: Improve MNIST with convolutions. I have made multiple submissions. On some submissions one or two of, reshape_and_normalize and/or mycallback and/or covolutional_model are graded and the other(s) not graded. I have attached screenshots here
Please click my name and message your notebook as an attachment.
I don’t see a problem with your notebook. Please submit one more time now. If that doesn’t work, reach out to coursera help.
I have reached out to Cousera and they asked me to reach out to the DL community. Hence, that’s why I made the post.
Moving forward, it’d help if you shared the fact that you had already reached out to coursera help center when creating the topic. I’ve now notified the staff about this issue.
Sure. Well noted. Thank you. I’ll be waiting for a response from the staff then
Hi Joachim! Welcome to the community! It seems the grader has issues with the model size. Please try reducing the number of units in the Conv2D and Dense layers. We added a unit test to tell you if the model can be submitted to the grader. You should see it when you re-open the notebook. You can copy your old solutions into the new notebook, then modify the number of units there. Remember to press the Submit button from the new notebook instead of the old one because it has the filename passed on to the grader. Hope it works!
It worked now. Thanks a lot.
Hello @chris.favila i still have the same grader error even though , i reduced the number of units in the Conv2D and Dense layers
- Introduction to Machine Learning with TensorFlow.js
- Neural Networks
Improving MNIST
Let's get physical.
Sometimes, nothing beats holding a copy of a book in your hands. Writing in the margins, highlighting sentences, folding corners. So this book is also available from Amazon as a paperback.
Buy now on Amazon
Input Normalization
Data augmentation, different algorithms.
We’ve covered a few different methods of solving MNIST in this chapter. It’s a great solid introduction to Machine Learning, but it’s also just the beginning. You may have noticed that although the accuracy from the test data is relatively high, 90%+, the real-world accuracy from recognizing your hand-drawn digits is much lower.
In this lecture, I’ll discuss some methods you may want to employ to improve your model’s performance.
The model was trained on a set of very normalized images; the shape and orientation of the handwritten digits are always the same. It’s written with a pen, whereas you will probably be trying to write with a mouse or track-pad, so your hand-drawn digits don’t quite look the same as the images in the data set.
One way you can try to improve the result is to make sure the input image is as close to the trained data as possible.
Center the digit
Crop it to a square bounding box and then scaling up
Rotate the drawing, so the digit is always vertical
The solution above talks about how to change the input data to make it look as close as possible to the training data. On the flip side, we can turn the training data to make it more representative of the various ways people can write digits. This is called data augmentation .
Take your source data set, and for each image, create slightly different versions.
You can try scaling the digits smaller and larger.
You can try shifting them a few pixels left and right.
You can try rotating them slightly clockwise and anticlockwise.
You can try corrupting the images somewhat, delete pixels, add a pixel in other places.
You can try to change the color or gray-scale values.
There are lots of ways you can adjust an image. Try to make the adjustments something that will be likely to happen in real life with the application rather than random, e.g. flipping by 90 degrees is perhaps unlikely with the application we are building here.
Create a new data set, much larger than the first with the images adjusted somehow, and run the model with the new data-set.
There are some helpful tools out there that can help create these adjusted images, imgaug [ 1 ] is one of them, but if you search for Data Augmentation Tools Machine Learning or something along those lines, you will find others.
There are many many different algorithms and collections of layers and methods you can use to solve MNIST .
Kaggle.com is an online platform where people compete to solve ML problems, like MNIST , to score the highest accuracy. There is a fantastic discussion [ 2 ] on Kaggle about the different algorithms you can use and the maximum accuracy that can be achieved for MNIST . I recommend you have a read of that it might give you some ideas for other things you can try.
Advanced JavaScript
This unique course teaches you advanced JavaScript knowledge through a series of interview questions. Bring your JavaScript to the 2021's today .
Level up your JavaScript now!
Subscribe to the PwC Newsletter
Join the community, edit social preview.
Add a new code entry for this paper
Remove a code repository from this paper.
Mark the official implementation from paper authors
Add a new evaluation result row.
- IMAGE CLASSIFICATION
- TRANSLATION
Remove a task
Add a method
- BATCH NORMALIZATION
- CONVOLUTION
Remove a method
- BATCH NORMALIZATION -
- CONVOLUTION -
Edit Datasets
An ensemble of simple convolutional neural network models for mnist digit recognition.
12 Aug 2020 · Sanghyeon An , Minjun Lee , Sanglee Park , Heerin Yang , Jungmin So · Edit social preview
We report that a very high accuracy on the MNIST test set can be achieved by using simple convolutional neural network (CNN) models. We use three different models with 3x3, 5x5, and 7x7 kernel size in the convolution layers. Each model consists of a set of convolution layers followed by a single fully connected layer. Every convolution layer uses batch normalization and ReLU activation, and pooling is not used. Rotation and translation is used to augment training data, which is frequently used in most image classification tasks. A majority voting using the three models independently trained on the training data set can achieve up to 99.87% accuracy on the test set, which is one of the state-of-the-art results. A two-layer ensemble, a heterogeneous ensemble of three homogeneous ensemble networks, can achieve up to 99.91% test accuracy. The results can be reproduced by using the code at: https://github.com/ansh941/MnistSimpleCNN
Code Edit Add Remove Mark official
Tasks edit add remove, datasets edit, results from the paper edit, methods edit add remove.
MNIST - Convolutions
First, we load the data using MLDatasets.jl :
The covariate data ( x ) were named 3 as these are three-dimensional arrays, containing the height x width x number of images. The training data are vectors indicating the digit.
SimpleChains' convolutional layers expect that we have a channels-in dimension, so we shape the images to be four dimensional It also currently defaults to 1-based indexing for its categories, so we shift all categories by 1.
We now define our model, LeNet5:
We define the inputs as being statically sized (28,28,1) images. Specifying the input sizes allows these to be checked. Making them static, which we can do either in our simple chain, or by adding static sizing to the images themselves using a package like StrideArrays.jl or HybridArrays.jl . These packages are recommended for allowing you to mix dynamic and static sizes; the batch size should probably be left dynamic, as you're unlikely to want to specialize code generation on this, given that it is likely to vary, increasing compile times while being unlikely to improve runtimes.
In SimpleChains , the parameters are not a part of the model, but live as a separate vector that you can pass around to optimizers of your choosing. If you specified the input size, you create a random initial parameter vector corresponding to the model:
The convolutional layers are initialized with a Glorot (Xavier) uniform distribution, while the dense layers are initialized with a Glorot (Xaviar) normal distribution. Biases are initialized to zero. Because the number of parameters can be a function of the input size, these must be provided if you didn't specify input dimension. For example:
To allow training to use multiple threads, you can create a gradient matrix, with a number of rows equal to the length of the parameter vector p , and one column per thread. For example:
Here, we're estimating that the number of physical cores is half the number of threads on an x86_64 system, which is true for most – but not all!!! – of them. Otherwise, we're assuming it is equal to the number of threads. This is of course also likely to be wrong, e.g. recent Power CPUs may have 4 or even 8 threads per core. You may wish to change this, or use Hwloc.jl for an accurate number.
Now that this is all said and done, we can train for 10 epochs using the ADAM optimizer with a learning rate of 3e-4 , and then assess the accuracy and loss of both the training and test data:
Training for an extra 10 epochs should be fast on most systems. Performance is currently known to be poor on the M1 (PRs welcome, otherwise we'll look into this eventually), but should be good/great on systems with AVX2/AVX512:
This document was generated with Documenter.jl version 0.27.25 on Thursday 13 July 2023 . Using Julia version 1.9.2.
Eduardo Avelar
C1w1 assignment: housing prices, c1w1 assignment: housing prices #.
https-deeplearning-ai/ tensorflow-1-public /C1/W1/assignment/ C1W1_Assignment.ipynb
Commit f16e408 on May 3, 2022, Compare
base cost 50k
50k each bedroom
1 bedroom 100k
scale 100,000 to 1
IMAGES
VIDEO
COMMENTS
We looked at how would improve Fashion MNIST using Convolutions. For this exercise see if we can improve MNIST to 99.5% accuracy or more by adding only a single convolutional layer and a single MaxPooling 2D layer to the model . We should stop training once the accuracy goes above this amount. It should happen in less than 10 epochs, so it's ...
C1W1 Assignment: Housing Prices C1W2: Implementing Callbacks in TensorFlow using the MNIST Dataset C1W3: Improve MNIST with Convolutions C1W4: Handling Complex Images - Happy or Sad Dataset C2W1: Using CNN's with the Cats vs Dogs Dataset C2W2: Tackle Overfitting with Data Augmentation
About. Improve MNIST with Convolution : how to enhance the Fashion MNIST neural network with convolutions to make it more accurate ! Resources
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window.
Exercise 3 -Solution. In the blogs you looked at how you would improve Fashion MNIST using Convolutions. For your exercise see if you can improve MNIST to 99.8% accuracy or more using only a single convolutional layer and a single MaxPooling 2D. You should stop training once the accuracy goes above this amount.
Exercise 3. In the videos you looked at how you would improve Fashion MNIST using Convolutions. For your exercise see if you can improve MNIST to 99.8% accuracy or more using only a single convolutional layer and a single MaxPooling 2D. You should stop training once the accuracy goes above this amount. It should happen in less than 20 epochs ...
1: The first layer of the convolutional neural network plays a dual role; it is both the input layer of the neural network and a layer that performs the first convolution operation on the input.
training_images=training_images / 255.0. val_images = val_images.reshape(10000, 28, 28, 1) val_images=val_images/255.. Next is to define your model. Now instead of the input layer at the top, you're going to add a Convolution. The parameters are: The number of convolutions you want to generate.
In this assignment, you will use the MNIST dataset. It consists of a training set of 60,000 handwritten digits with corresponding labels, and a test set of 10,000 images. The images have been normalised and centred. The dataset is frequently used in machine learning research, and has become a standard benchmark for image classification models.
Programming Assignment: Exercise 3 (Improve MNIST with convolutions) Week 4 - Using Real-world Images. Programming Assignment: Exercise 4 (Handling complex images) 2. Convolutional Neural Networks in TensorFlow. ... Programming Assignment: Fréchet Inception Distance; Week 2 - GAN Disadvantages and Bias. Lab: Alternatives: Variational ...
MNIST - Convolutions. The covariate data ( x) were named 3 as these are three-dimensional arrays, containing the height x width x number of images. The training data are vectors indicating the digit. SimpleChains' convolutional layers expect that we have a channels-in dimension, so we shape the images to be four dimensional It also currently ...
Moving forward, it'd help if you shared the fact that you had already reached out to coursera help center when creating the topic. I've now notified the staff about this issue.
One way you can try to improve the result is to make sure the input image is as close to the trained data as possible. Center the digit. Figure 1. Centering a digit. Crop it to a square bounding box and then scaling up. Figure 2. Cropping a digit to a bounding box and then scaling.
For your exercise see if you can improve MNIST to 99.8% accuracy or more using only a single convolutional layer and a single MaxPooling 2D. You should stop training once the accuracy goes above this amount. It should happen in less than 20 epochs, so it's ok to hard code the number of epochs for training, but your training must end once it ...
We report that a very high accuracy on the MNIST test set can be achieved by using simple convolutional neural network (CNN) models. We use three different models with 3x3, 5x5, and 7x7 kernel size in the convolution layers. Each model consists of a set of convolution layers followed by a single fully connected layer. Every convolution layer ...
C1W1 Assignment: Housing Prices C1W2: Implementing Callbacks in TensorFlow using the MNIST Dataset C1W3: Improve MNIST with Convolutions C1W4: Handling Complex Images - Happy or Sad Dataset C2W1: Using CNN's with the Cats vs Dogs Dataset C2W2: Tackle Overfitting with Data Augmentation
nouran551/Improve-MNIST-with-convolutions. This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. master. Switch branches/tags. Branches Tags. Could not load branches. Nothing to show {{ refName }} default View all branches. Could not load tags.
MNIST - Convolutions. First, we load the data using MLDatasets.jl: # (28, 28, 60000) The covariate data ( x) were named 3 as these are three-dimensional arrays, containing the height x width x number of images. The training data are vectors indicating the digit. SimpleChains' convolutional layers expect that we have a channels-in dimension, so ...
C1W1 Assignment: Housing Prices C1W2: Implementing Callbacks in TensorFlow using the MNIST Dataset C1W3: Improve MNIST with Convolutions C1W4: Handling Complex Images - Happy or Sad Dataset C2W1: Using CNN's with the Cats vs Dogs Dataset C2W2: Tackle Overfitting with Data Augmentation
C1W1 Assignment: Housing Prices C1W2: Implementing Callbacks in TensorFlow using the MNIST Dataset C1W3: Improve MNIST with Convolutions C1W4: Handling Complex Images - Happy or Sad Dataset C2W1: Using CNN's with the Cats vs Dogs Dataset C2W2: Tackle Overfitting with Data Augmentation
Saved searches Use saved searches to filter your results more quickly
C1W1 Assignment: Housing Prices C1W2: Implementing Callbacks in TensorFlow using the MNIST Dataset C1W3: Improve MNIST with Convolutions C1W4: Handling Complex Images - Happy or Sad Dataset C2W1: Using CNN's with the Cats vs Dogs Dataset C2W2: Tackle Overfitting with Data Augmentation