Django users and superusers not preserved in my dockerized app

  database, django, docker, postgresql, python

I am developing a Django app using Docker, and postgreSQL as my database.

I build my docker image and everything is working fine except that my superuser and my users are not being preserved if I run:

sudo docker-compose down
sudo docker-compose up -d

If right after I run sudo docker-compose up I go to my localhost in the browser I obtain an error and the output of docker-compose logs looks as follows :

db_1   | The files belonging to this database system will be owned by user "postgres".
db_1   | This user must also own the server process.
db_1   | 
db_1   | The database cluster will be initialized with locale "en_US.utf8".
db_1   | The default database encoding has accordingly been set to "UTF8".
db_1   | The default text search configuration will be set to "english".
db_1   | 
db_1   | Data page checksums are disabled.
db_1   | 
db_1   | fixing permissions on existing directory /var/lib/postgresql/data ... ok
db_1   | creating subdirectories ... ok
db_1   | selecting dynamic shared memory implementation ... posix
db_1   | selecting default max_connections ... 100
db_1   | selecting default shared_buffers ... 128MB
db_1   | selecting default time zone ... Etc/UTC
db_1   | creating configuration files ... ok
db_1   | running bootstrap script ... ok
db_1   | performing post-bootstrap initialization ... ok
db_1   | syncing data to disk ... ok
db_1   | 
db_1   | 
db_1   | Success. You can now start the database server using:
db_1   | 
db_1   |     pg_ctl -D /var/lib/postgresql/data -l logfile start
db_1   | 
db_1   | initdb: warning: enabling "trust" authentication for local connections
db_1   | You can change this by editing pg_hba.conf or using the option -A, or
db_1   | --auth-local and --auth-host, the next time you run initdb.
db_1   | waiting for server to start....2021-02-23 14:29:45.945 UTC [48] LOG:  starting PostgreSQL 13.1 (Debian 13.1-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
db_1   | 2021-02-23 14:29:45.946 UTC [48] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1   | 2021-02-23 14:29:45.949 UTC [49] LOG:  database system was shut down at 2021-02-23 14:29:45 UTC
db_1   | 2021-02-23 14:29:45.952 UTC [48] LOG:  database system is ready to accept connections
db_1   |  done
db_1   | server started
db_1   | 
db_1   | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
db_1   | 
db_1   | waiting for server to shut down...2021-02-23 14:29:46.067 UTC [48] LOG:  received fast shutdown request
db_1   | .2021-02-23 14:29:46.069 UTC [48] LOG:  aborting any active transactions
db_1   | 2021-02-23 14:29:46.069 UTC [48] LOG:  background worker "logical replication launcher" (PID 55) exited with exit code 1
db_1   | 2021-02-23 14:29:46.070 UTC [50] LOG:  shutting down
db_1   | 2021-02-23 14:29:46.079 UTC [48] LOG:  database system is shut down
db_1   |  done
db_1   | server stopped
db_1   | 
db_1   | PostgreSQL init process complete; ready for start up.
db_1   | 
db_1   | 2021-02-23 14:29:46.182 UTC [1] LOG:  starting PostgreSQL 13.1 (Debian 13.1-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
db_1   | 2021-02-23 14:29:46.183 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
db_1   | 2021-02-23 14:29:46.183 UTC [1] LOG:  listening on IPv6 address "::", port 5432
db_1   | 2021-02-23 14:29:46.185 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1   | 2021-02-23 14:29:46.188 UTC [67] LOG:  database system was shut down at 2021-02-23 14:29:46 UTC
db_1   | 2021-02-23 14:29:46.191 UTC [1] LOG:  database system is ready to accept connections
db_1   | 2021-02-23 14:29:52.701 UTC [75] ERROR:  relation "django_session" does not exist at character 109
db_1   | 2021-02-23 14:29:52.701 UTC [75] STATEMENT:  SELECT "django_session"."session_key", "django_session"."session_data", "django_session"."expire_date" FROM "django_session" WHERE ("django_session"."expire_date" > '2021-02-23T14:29:52.696886+00:00'::timestamptz AND "django_session"."session_key" = 'e85xghsfybtlpfeq9vs1pbinfegm9z03')
db_1   | 2021-02-23 14:29:52.719 UTC [75] ERROR:  relation "django_session" does not exist at character 109
db_1   | 2021-02-23 14:29:52.719 UTC [75] STATEMENT:  SELECT "django_session"."session_key", "django_session"."session_data", "django_session"."expire_date" FROM "django_session" WHERE ("django_session"."expire_date" > '2021-02-23T14:29:52.718768+00:00'::timestamptz AND "django_session"."session_key" = 'e85xghsfybtlpfeq9vs1pbinfegm9z03')
db_1   | 2021-02-23 14:29:52.720 UTC [75] ERROR:  relation "django_session" does not exist at character 109
db_1   | 2021-02-23 14:29:52.720 UTC [75] STATEMENT:  SELECT "django_session"."session_key", "django_session"."session_data", "django_session"."expire_date" FROM "django_session" WHERE ("django_session"."expire_date" > '2021-02-23T14:29:52.719737+00:00'::timestamptz AND "django_session"."session_key" = 'e85xghsfybtlpfeq9vs1pbinfegm9z03')
db_1   | 2021-02-23 14:29:52.721 UTC [75] ERROR:  relation "django_session" does not exist at character 109
db_1   | 2021-02-23 14:29:52.721 UTC [75] STATEMENT:  SELECT "django_session"."session_key", "django_session"."session_data", "django_session"."expire_date" FROM "django_session" WHERE ("django_session"."expire_date" > '2021-02-23T14:29:52.720615+00:00'::timestamptz AND "django_session"."session_key" = 'e85xghsfybtlpfeq9vs1pbinfegm9z03')
db_1   | 2021-02-23 14:29:52.726 UTC [75] ERROR:  relation "django_session" does not exist at character 109
db_1   | 2021-02-23 14:29:52.726 UTC [75] STATEMENT:  SELECT "django_session"."session_key", "django_session"."session_data", "django_session"."expire_date" FROM "django_session"  LIMIT 21
db_1   | 2021-02-23 14:29:52.726 UTC [75] ERROR:  relation "django_session" does not exist at character 109
db_1   | 2021-02-23 14:29:52.726 UTC [75] STATEMENT:  SELECT "django_session"."session_key", "django_session"."session_data", "django_session"."expire_date" FROM "django_session" WHERE ("django_session"."expire_date" > '2021-02-23T14:29:52.696886+00:00'::timestamptz AND "django_session"."session_key" = 'e85xghsfybtlpfeq9vs1pbinfegm9z03')  LIMIT 21
db_1   | 2021-02-23 14:29:52.727 UTC [75] ERROR:  relation "django_session" does not exist at character 109
db_1   | 2021-02-23 14:29:52.727 UTC [75] STATEMENT:  SELECT "django_session"."session_key", "django_session"."session_data", "django_session"."expire_date" FROM "django_session" WHERE ("django_session"."expire_date" > '2021-02-23T14:29:52.696886+00:00'::timestamptz AND "django_session"."session_key" = 'e85xghsfybtlpfeq9vs1pbinfegm9z03')  LIMIT 21
db_1   | 2021-02-23 14:29:52.727 UTC [75] ERROR:  relation "django_session" does not exist at character 109
db_1   | 2021-02-23 14:29:52.727 UTC [75] STATEMENT:  SELECT "django_session"."session_key", "django_session"."session_data", "django_session"."expire_date" FROM "django_session" WHERE ("django_session"."expire_date" > '2021-02-23T14:29:52.696886+00:00'::timestamptz AND "django_session"."session_key" = 'e85xghsfybtlpfeq9vs1pbinfegm9z03')  LIMIT 21
db_1   | 2021-02-23 14:29:52.728 UTC [75] ERROR:  relation "django_session" does not exist at character 109
db_1   | 2021-02-23 14:29:52.728 UTC [75] STATEMENT:  SELECT "django_session"."session_key", "django_session"."session_data", "django_session"."expire_date" FROM "django_session" WHERE ("django_session"."expire_date" > '2021-02-23T14:29:52.696886+00:00'::timestamptz AND "django_session"."session_key" = 'e85xghsfybtlpfeq9vs1pbinfegm9z03')  LIMIT 21
db_1   | 2021-02-23 14:29:52.729 UTC [75] ERROR:  relation "django_session" does not exist at character 109
db_1   | 2021-02-23 14:29:52.729 UTC [75] STATEMENT:  SELECT "django_session"."session_key", "django_session"."session_data", "django_session"."expire_date" FROM "django_session" WHERE ("django_session"."expire_date" > '2021-02-23T14:29:52.728823+00:00'::timestamptz AND "django_session"."session_key" = 'e85xghsfybtlpfeq9vs1pbinfegm9z03')
web_1  | Watching for file changes with StatReloader
web_1  | Performing system checks...
web_1  | 
web_1  | System check identified no issues (0 silenced).
web_1  | 
web_1  | You have 18 unapplied migration(s). Your project may not work properly until you apply the migrations for app(s): accounts, admin, auth, contenttypes, sessions.
web_1  | Run 'python manage.py migrate' to apply them.
web_1  | February 23, 2021 - 14:29:46
web_1  | Django version 2.2.19, using settings 'config.settings'
web_1  | Starting development server at http://0.0.0.0:8000/
web_1  | Quit the server with CONTROL-C.
web_1  | Internal Server Error: /
web_1  | Traceback (most recent call last):
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/contrib/sessions/backends/base.py", line 189, in _get_session
web_1  |     return self._session_cache
web_1  | AttributeError: 'SessionStore' object has no attribute '_session_cache'
web_1  | 
web_1  | During handling of the above exception, another exception occurred:
web_1  | 
web_1  | Traceback (most recent call last):
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
web_1  |     return self.cursor.execute(sql, params)
web_1  | psycopg2.errors.UndefinedTable: relation "django_session" does not exist
web_1  | LINE 1: ...ession_data", "django_session"."expire_date" FROM "django_se...
web_1  |                                                              ^
web_1  | 
web_1  | 
web_1  | The above exception was the direct cause of the following exception:
web_1  | 
web_1  | Traceback (most recent call last):
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/core/handlers/exception.py", line 34, in inner
web_1  |     response = get_response(request)
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/core/handlers/base.py", line 145, in _get_response
web_1  |     response = self.process_exception_by_middleware(e, request)
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/core/handlers/base.py", line 143, in _get_response
web_1  |     response = response.render()
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/template/response.py", line 106, in render
web_1  |     self.content = self.rendered_content
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/template/response.py", line 83, in rendered_content
web_1  |     content = template.render(context, self._request)
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/template/backends/django.py", line 61, in render
web_1  |     return self.template.render(context)
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/template/base.py", line 171, in render
web_1  |     return self._render(context)
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/template/base.py", line 163, in _render
web_1  |     return self.nodelist.render(context)
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/template/base.py", line 937, in render
web_1  |     bit = node.render_annotated(context)
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/template/base.py", line 904, in render_annotated
web_1  |     return self.render(context)
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/template/loader_tags.py", line 150, in render
web_1  |     return compiled_parent._render(context)
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/template/base.py", line 163, in _render
web_1  |     return self.nodelist.render(context)
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/template/base.py", line 937, in render
web_1  |     bit = node.render_annotated(context)
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/template/base.py", line 904, in render_annotated
web_1  |     return self.render(context)
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/template/loader_tags.py", line 62, in render
web_1  |     result = block.nodelist.render(context)
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/template/base.py", line 937, in render
web_1  |     bit = node.render_annotated(context)
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/template/base.py", line 904, in render_annotated
web_1  |     return self.render(context)
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/template/defaulttags.py", line 302, in render
web_1  |     match = condition.eval(context)
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/template/defaulttags.py", line 876, in eval
web_1  |     return self.value.resolve(context, ignore_failures=True)
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/template/base.py", line 671, in resolve
web_1  |     obj = self.var.resolve(context)
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/template/base.py", line 796, in resolve
web_1  |     value = self._resolve_lookup(context)
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/template/base.py", line 829, in _resolve_lookup
web_1  |     current = current[bit]
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/utils/functional.py", line 256, in inner
web_1  |     self._setup()
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/utils/functional.py", line 392, in _setup
web_1  |     self._wrapped = self._setupfunc()
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/contrib/auth/middleware.py", line 24, in <lambda>
web_1  |     request.user = SimpleLazyObject(lambda: get_user(request))
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/contrib/auth/middleware.py", line 12, in get_user
web_1  |     request._cached_user = auth.get_user(request)
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/contrib/auth/__init__.py", line 182, in get_user
web_1  |     user_id = _get_user_session_key(request)
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/contrib/auth/__init__.py", line 59, in _get_user_session_key
web_1  |     return get_user_model()._meta.pk.to_python(request.session[SESSION_KEY])
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/contrib/sessions/backends/base.py", line 54, in __getitem__
web_1  |     return self._session[key]
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/contrib/sessions/backends/base.py", line 194, in _get_session
web_1  |     self._session_cache = self.load()
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/contrib/sessions/backends/db.py", line 43, in load
web_1  |     s = self._get_session_from_db()
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/contrib/sessions/backends/db.py", line 32, in _get_session_from_db
web_1  |     return self.model.objects.get(
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/db/models/manager.py", line 82, in manager_method
web_1  |     return getattr(self.get_queryset(), name)(*args, **kwargs)
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/db/models/query.py", line 402, in get
web_1  |     num = len(clone)
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/db/models/query.py", line 256, in __len__
web_1  |     self._fetch_all()
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/db/models/query.py", line 1242, in _fetch_all
web_1  |     self._result_cache = list(self._iterable_class(self))
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/db/models/query.py", line 55, in __iter__
web_1  |     results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1142, in execute_sql
web_1  |     cursor.execute(sql, params)
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 99, in execute
web_1  |     return super().execute(sql, params)
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 67, in execute
web_1  |     return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
web_1  |     return executor(sql, params, many, context)
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
web_1  |     return self.cursor.execute(sql, params)
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/db/utils.py", line 89, in __exit__
web_1  |     raise dj_exc_value.with_traceback(traceback) from exc_value
web_1  |   File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
web_1  |     return self.cursor.execute(sql, params)
web_1  | django.db.utils.ProgrammingError: relation "django_session" does not exist
web_1  | LINE 1: ...ession_data", "django_session"."expire_date" FROM "django_se...
web_1  |                                                              ^
web_1  | 
web_1  | [23/Feb/2021 14:29:52] "GET / HTTP/1.1" 500 259243

I resolve this by running
sudo docker-compose exec web python manage.py migrate, at which point my app works and I can create users and superusers. However, if I run the steps above again, i.e. do compose down, compose up and migrate, my superuser and my users are no longer there.

How do I set up my database/user model so that my users and superusers live on when I restart/rebuild my docker image?

My thinking was that the users would be saved in the docker volume specified in my docker-compose.yml file. Also I don’t quite understand why I must migrate everytime I compose up, that also seems like a problem to me.

I am using django 2.2 and psycopg 2.8.5. Here follow what I think are the relevant files. Please let me know if additional information would be useful. Thanks a lot in advance!

docker-compose.yml:

version: '3.8'

volumes:
  postgres_data:
    driver: local

services:
  db:
    image: postgres:13
    volumes:
      - postgres_data:/var/lib/posgresql/data/
    environment:
      - "POSTGRES_HOST_AUTH_METHOD=trust"

  web:
    build: .
    command: python3 /code/manage.py runserver 0.0.0.0:8000
    volumes:
      - .:/code
    ports:
      - 8000:8000
    depends_on: 
      - db

Dockerfile:

# Pull base image
FROM python:3.8

# Set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1

# set work directory
WORKDIR /code

# Install dependencies
COPY Pipfile Pipfile.lock /code/
RUN pip install pipenv && pipenv install --system 

# Copy project

COPY . /code/

settings.py:

"""
Django settings for config project.

Generated by 'django-admin startproject' using Django 2.2.19.

For more information on this file, see
https://docs.djangoproject.com/en/2.2/topics/settings/

For the full list of settings and their values, see
https://docs.djangoproject.com/en/2.2/ref/settings/
"""

import os

# Build paths inside the project like this: os.path.join(BASE_DIR, ...)
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))


# Quick-start development settings - unsuitable for production
# See https://docs.djangoproject.com/en/2.2/howto/deployment/checklist/

# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = '<key>'

# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = True

# Mapbox
MAPBOX_TOKEN = '<token>'


ALLOWED_HOSTS = []
AUTH_USER_MODEL = 'accounts.CustomUser'
LOGIN_REDIRECT_URL = 'assets'
LOGOUT_REDIRECT_URL = 'login'


# Application definition

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',

    # Local
    'accounts',
    'assets',
]

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
]

ROOT_URLCONF = 'config.urls'

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [os.path.join(BASE_DIR, 'templates')],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
        },
    },
]

WSGI_APPLICATION = 'config.wsgi.application'


# Database
# https://docs.djangoproject.com/en/2.2/ref/settings/#databases

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql_psycopg2',
        'NAME': 'postgres',
        'USER': 'postgres',
        'PASSWORD' : 'postgres',
        'HOST': 'db',
        'PORT': 5432
    }
}


# Password validation
# https://docs.djangoproject.com/en/2.2/ref/settings/#auth-password-validators

AUTH_PASSWORD_VALIDATORS = [
    {
        'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
    },
]


# Internationalization
# https://docs.djangoproject.com/en/2.2/topics/i18n/

LANGUAGE_CODE = 'en-us'

TIME_ZONE = 'UTC'

USE_I18N = True

USE_L10N = True

USE_TZ = True


# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/2.2/howto/static-files/

STATIC_URL = '/static/'
STATICFILES_DIRS = (str(os.path.join(BASE_DIR, 'static')),)
STATIC_ROOT = str(os.path.join(BASE_DIR, 'staticfiles'))

STATICFILES_FINDERS = [
    "django.contrib.staticfiles.finders.FileSystemFinder",
    "django.contrib.staticfiles.finders.AppDirectoriesFinder",
]

Output of sudo docker ps:

CONTAINER ID   IMAGE           COMMAND                  CREATED          STATUS          PORTS                    NAMES
0268c7194003   platform2_web   "python3 /code/manag…"   35 minutes ago   Up 35 minutes   0.0.0.0:8000->8000/tcp   platform2_web_1
d7935baa3337   postgres:13     "docker-entrypoint.s…"   35 minutes ago   Up 35 minutes   5432/tcp                 platform2_db_1

Source: Python Questions

LEAVE A COMMENT