When I tried to run this command: python manage.py createsuperuser in my Django project, it gave me a super long error which included the following:
File “manage.py”, line 22, in main() File “manage.py”, line 18, in main execute_from_command_line(sys.argv) File “/Users/username/.local/share/virtualenvs/django-learn-csZU2bYZ/lib/python3.8/site-packages/django/core/management/init.py”, line 364, in execute_from_command_line utility.execute()
….
ImportError: cannot import name ‘path’ from ‘django.urls’ (/Users/username/.local/share/virtualenvs/django-learn-csZU2bYZ/lib/python3.8/site-packages/django/urls/init.py)
Solution
The problem in my case was that I had the wrong version of Django installed. In order to run the python manage.py createsuperuser command you need at least Django 2.
Run this command to check the version of Django you are currently working with:
python -m django --version
Run this command to install the latest version of Django.
pip install --upgrade django
Run the python -m django --version command again and make sure your new Django version is 2.x or above. If so, you should be able to run python manage.py createsuperuser now
Django is a high-level Python web framework that enables rapid development of secure and maintainable websites. It is one of the most popular Python backend development frameworks. It is known for its security, simplicity, and reliability. It is without a doubt one of the best frameworks to learn.
Note: You must have Python 3 or later installed on your computer to complete this tutorial.
Create a new empty project. The first thing we need to do is install Django in our new project. You might be tempted to do it using the command pip install django, but I would advise against this because it installs the Django package on your global system rather than just in this specific project.
Instead, you should use a tool called pipenv, which allows you to set up a virtual environment for each Python project that you create on your system. Install it globally by running this command:
pip install pipenv
With pipenv installed, run this command in your project folder to create a virtual environment.
pipenv shell
Any packages we install, including Django, will be installed into this virtual environment rather than on our global system. This command will also generate a Pipfile in your project directory. This Pipfile will list any packages that you install in the virtual environment.
Run this command to install Django in the virtual environment
pipenv install django
In Django, a project is basically the overall website/application. Then, you have the concept of apps; there are usually multiple apps in each project.
Run this command to create a new project (put your project’s name in place of project-name):
django-admin startproject project-name
cd project-name
Inside your new project folder, you should see a second folder with the same project name and a manage.py file. You rarely change any of the code in the manage.py file, but you will use this file all the time. We use this file to run the server, create migrations, and carry out many other commands.
Once inside the project folder we just created, run this command to run the server on http: 127.0.0.1:8000 (port 8000):
python manage.py runserver
# ctrl+c to stop
To run the server on a different port
python manage.py runserver 8081
You will probably get the below error. The cause of the error is that there are migrations ready to create the default, necessary database tables that Django uses that have not been applied.
To apply them, we run this command:
python manage.py migrate
Now, your server should be up and running with no errors. If you type localhost:8000 (or whichever port your project is running on) in your browser, you should see this default Django page.
You’ve officially created your first Django project. The next step is to start adding apps to your project. Stay tuned for a tutorial on that.
Web scraping is the process of automatically extracting data from a website.
You will need an understanding of basic HTML page structure in order to grasp this tutorial. Web scraping with BeautifulSoup to get data/text from a page is done by referencing specific HTML semantic tags, classes, and ids on that page and getting the data from within them. You will better understand what I mean as we continue.
BeautifulSoup is the Python package we are going to use to do the web scraping. Requests is a package that allows you to send HTTP/1.1 requests extremely easily.
In this tutorial, we are going to be parsing the information from the daily forecast element on the Orlando, FL weather page on weather.com (this is the link). We will transfer the information to a structured, table format using Pandas.
The picture below shows the div element which holds the daily forecast information for the week. When you view this tutorial, the numbers will probably not be the same but, other than that, you should see something that looks like this. This is the element we are going to be scraping.
In Terminal, create a new folder with mkdir, cd into the folder, and create a .py file. I will call mine scraper.py. This file will contain the code for the scraper.
In the next lines, we are installing some necessary dependencies using pip, the Python package manager, into our project folder.
This line uses the request package to get the HTML page of the URL in quotes. In this case, we are using a page on the site.
Next, write this line:
soup = BeautifulSoup(page.content, 'html.parser')
This line initializes BeautifulSoup in our project. Now, we can use the soup variable to access the functions within the BeautifulSoup package. The first parameter in the statement is what you want to scrape (in this case, the content of the page we got earlier) and the second parameter just tells BeautifulSoup that it will be parsing HTML.
Now, if you are using Google Chrome, you can Inspect the page and find the id of the div element which holds the data you want to scrape.
As you can see from the above image, I found that the div containing the Daily Forecast information had an id of “WxuDailyWeatherCard-main-bb1a17e7-dc20-421a-b1b8-c117308c6626”
After getting the id of the div element which holds the data we want, we are defining a reference to the div element with the following line:
Now, we can get access to the data within the items array. We will make an array of the dates within the items array using this line:
date = [item.find('h3', class_='Column--label--L3RrD').get_text() for item in items]
Using the same process as before, I found that all the dates were stored in a h3 with a class of ‘Column–label–L3RrD’. The above line creates an array of dates by running the find() method on every item within the items array.
Now, we can do this for the temperature and chance of rain data stored in the items array.
temp = [item.find(class_='Column--temp--2v_go').get_text() for item in items]
chanceOfRain = [item.find(class_='Column--precip--2H5Iw').get_text() for item in items]
Now, you can access the date, temperature, and chance of rain data with the variables we just created!
Pandas Addition
As an extra, you can put the weather data we just scraped into a simple Panda’s data frame so that it looks better.
First, install the Pandas package and import Pandas at the top of your scraper.py file.
import pandas as pd
Under the data definitions, write this line. This line creates a data frame in the variable weather_info with the data.
Using the Terminal or Command Line will prove to be a super handy and important aspect of your software development process. Here are a few commands that will most definitely be useful for you when using the Terminal. I’m using a Mac.
Make a new folder
mkdir folder-name
Go into folder (only up 1 level)
cd folder-name
To cd into a folder with a space in the name, use the backslash character between the words
cd Software\ Development
Go backwards by one level in the folder system
cd ..
Go to your computer’s root directory
cd ~
Clear the Terminal window (just for declutter)
clear
See the contents of a folder
ls
See the hidden files in a folder
ls -a
Copy file to a different directory. This line copies the file file-name.txt to the directory/file one above the current directory/file.
cp file-name.txt ../
Rename a file
mv file-name.txt NEW-FILE-NAME.txt
Remove file from current directory
rm file-name.txt
Shows where an application lies in your file system. In this example, we are seeing the location of the Git application. This command will either return the file system location if the application exists on your machine or no output if there is no application with that name on your machine
which application-name
which git
Shows where you are currently in your file system
pwd
Basically a print statement but in the Terminal
echo hello
Make a new file and put the defined text inside of it. In this case, we put the text hello in a file called hello.txt
Create a new Heroku app by either clicking “Create a new app” in the center of the screen or “New” at the top right of the screen.
Choose an App Name and the Region (either the US or Europe) for your new app. The name can be anything you want. Then, just click the Create app button.
Step 3: Add a PostgreSQL Database
To attach a PostgreSQL database to the blank app you just made, you need to go to the Resources tab in the header of the new app’s dahsboard. Then type Heroku Postgres into the Add-ons search field. When shown, select the suggested Heroku Postgres add-on from the dropdown:
The next popup asks you to choose a pricing plan for the database. Select the Hobby Dev – Free plan and click Provision.
Now your PostgreSQL database is up !
Step 4: Get Heroku DB Information
To find the credentials and the connection URL for the PostgreSQL database, make sure you’re still in the Resources tab and click on the Heroku Postgres resource we just added.
That should bring you to this screen:
Select the Settings tab in the header of that screen. You will be navigated to a page where Here, you can click the View Credentials button to see the credentials of your PostgreSQL database.
You will need these values to input into your PgAdmin interface
Step 5: Configure PgAdmin w/ Heroku DB Credentials
Open up PgAdmin
Right-click on Servers at the very top of the left-side panel
Choose Create > Serve
Fill out the new server form with all of the corresponding information from the Heroku View Credentials page.
After you save the new server, it will become populated with a huge list of databases. You will not have access to any of them except the one with your Database name from Heroku. Scroll through and find this one.
Then, as you would with any other PgAdmin database, go to Schemas > public > Create new Table. Populate this table with the fields you want.
This tutorial assumes that you’ve already set up a basic Node.js project with Express. I have other tutorials explaining how to do this if you don’t know how.
First, create a new python file in your Node project directory. I will call mine hello.py. In hello.py, I will write a simple Python print statement to test.
print("Hello world")
In my main JavaScript file, I’m going to add the bolded line:
const express = require('express');
const { spawn } = require('child_process');
const app = express();
app.get('/', (req, res) => {
console.log('Hello world');
});
app.listen(4000, console.log('Server started on port 4000'));
In the code above, we set up a basic Express app with one get router. Pay attention to the bolded line:
const { spawn } = require('child_process');
Next, we will use the new spawn class that we imported with this line from the child_process library.
The first parameter is the program we want to call and the second is an array of strings that will use the python program. It is the same command if you wanted to write it in a shell to run the script1.py: python 'script1.py'
The data argument that we are receiving from the callback function will be the output of the Python code we wrote in hello.py. We could also get access to the outputs this way. In the code below, we have to either return or console.log the dataToSend.
python.stdout.on('data', function (data) {
dataToSend = data.toString();
});
If this event fails, the error event will be called:
First, make sure you have Node installed. I’m not going to show how to do that in those tutorial. For developers that already know how to use Node, this tutorial can serve as a guide to help jog your memory whenever you forget all the steps to building a new Node.js project.
Once you have Node installed, you can write these commands in Terminal to get started:
mkdir folder-name (create a new folder)
npm init (initialize Node Package Manager)
touch index.js (create server file)
For the npm init step, it will make you click through a bunch of prompts/set-up questions to set up NPM in your folder.
For your server file, it doesn’t matter what it’s name is as long as you make sure it has the same name as the one you defined in the NPM initialization questions.
In Terminal within your project directory, add these NPM packages to your project using the command npm i or npm install
npm i express (install express - a Node web framework)
npm i -D nodemon (install nodemon as a dev dependency)
Open up your server file (index.js) in a code editor (I prefer Visual Studio Code) and type this Node code:
const express = require('express');
const app = express();
const PORT = process.env.PORT || 4000;
app.listen(PORT, console.log(`Server started on port ${PORT}`));
The Express framework is the most popular Node backend web framework by far and it will make working with Node for the web 100x easier.
To run this and make sure it works, type this command in your Terminal (make sure your in your project directory):
nodemon index.js
The Terminal should output the string “Server started on port 4000” (or port 5000, 6000, etc. based on what you put in your code).
This means that the project is running on the specified port on your localhost.
try {
await sequelize.authenticate();
console.log('Connection has been established successfully.');
} catch (error) {
console.error('Unable to connect to the database:', error);
}
Create a new model
The db variable is imported from the Sequelize connection above (Opt 1 or 2) which we are assuming is in a separate file.
This error was caused when I tried to locally run a server on a port that was already being used by another server that I had previously left running and forgot to turn off.
This command will show you all the process(es) running on a port and their PIDs
sudo lsof -i :3000
Kill the process by replacing <PID> with the process’ PID number
In the upload middleware, we define a key value. This key serves essentially as the image’s name in AWS, differentiating images in the list in the AWS Console. We are setting the key to the date at which the image is saved.