Roadmap

I’ve been thinking about how best to mature Starlette as it becomes more and more capable for large-scale projects. The key thing I want the framework to focus on is low-complexity of the underlying stack.

Rather than have a monolithic application object, where we put all the functionality on, I’d like to see us focus on independent components. I think this is really important, because it makes it so much easier to understand the stack that you’re building on top of. Components are independently testable, can be understood in isolation, and can be swapped out or adapted more easily.

One example here is we currently have template_dirs on the application instance. That makes it less clear what interactions the application instance might have with template dirs under the hood. Instead we should keep all the templating properly independent from other parts of the stack.

A related aspect of this is routing. Right now we promote the decorator style, which is convenient. I think we should move towards promoting a more explicit style made up of the actual BaseRoute instances that the Router class uses. The reason here is that it explicitly surfaces information about how the underlying stack function. If you want to implement an alternative to Route, WebSocketRoute, Mount, Host, it’s already more clear how you’d do that. Even the call stack that gets passed through is being made more explicit. “Look at this class implementation if you want to understand what’s happening here”.

Here’s a sketch of how a more explicit style would fit together:

settings.py

DEBUG = config(...)
DATABASE_URL = config(...)

resources.py

cache = Cache(...)
database = Database(...)
templates = Jinja2Templates(...)

routing.py

routes = [
    Route(...),
    Route(...),
    WebSocketRoute(...),
    Mount(...)
]

application.py

middleware = [
    ...
]

app = Starlette(routes=routes, middleware=...)

[tomchristie]

Did some sketching out today of what you need in order to start getting towards a rough Django-ish feature parity.

Notable bits missing at the moment…

• Password hashing.
• orm needs a ModelRegistry
• typesystem doesn’t yet have any API for validate_all, or for async validators.
• Allow RedirectResponse to take an endpoint name.
• Make request not be strictly required in TemplateResponse.
• Few remaining question marks over typesystem API.
• Need a general request.parse() function, that just handles whatever media type.

Even now, it’s quite an imposing chunk of code.

# settings.py
config = Config(env_file='.env')

ALLOWED_HOSTS = config('ALLOWED_HOSTS', cast=CommaSeperated, default="127.0.0.1,localhost")
TESTING = config('TESTING', cast=bool)
DATABASE_URL = config('DATABASE_URL', cast=databases.DatabaseURL, default='sqlite:///db.sqlite')
TEST_DATABASE_URL = DATABASE_URL.replace(name=f'test_{DATABASE_URL}')
SECRET_KEY = config('SECRET_KEY', cast=Secret)


# resources.py
templates = Jinja2Templates(directory='templates')
statics = StaticFiles(directory='statics', packages=['boostrap4'])
forms = typesystem.Jinja2Forms(package='bootstrap4')
database = databases.Database(url=TEST_DATABASE_URL if TESTING else DATABASE_URL)
models = orm.ModelRegistry(database=database)
hasher = PBKDF2PasswordHasher()


# models.py
class User(orm.Model):
    __table__ = 'users'
    __registry__ = models

    id = orm.Integer(primary_key=True)
    username = orm.String(max_length=200)
    password = orm.String(max_length=200)

    async def check_password(self, password):
        return await hasher.check_password(password, hashed=self.password)

    async def set_password(self, password):
        hashed = await hasher.make_password(password)


class Note(orm.Model):
    __table__ = 'notes'
    __registry__ = models

    id = orm.Integer(primary_key=True)
    user = orm.ForeignKey(User)
    text = orm.Text(default='')
    completed = orm.Boolean(default=False)


# schemas.py
class Login(typesystem.Schema):
    username = typesystem.String(max_length=200)
    password = typesystem.Password(max_length=200)

    async def validate_all(self, data):
        user = await User.objects.get_or_none(username=data['username'])
        if user is not None and await user.check_password(data['password']):
            return user
        raise ValidationError('Invalid login')


class CreateNote(typesystem.Schema):
    text = typesystem.Text()
    completed = typesystem.Boolean(default=False)


# endpoints.py
@requires(AUTHENTICATED, redirect='login')
async def homepage(request):
    if request.method == 'GET':
        form = forms.Form(CreateNote)
        notes = await Note.objects.all(user=request.user)
        return templates.TemplateResponse('login.html', {'form': form, 'notes': notes})

    data = await request.parse()
    validated, errors = await CreateNote().validate_or_error(data=data)
    if errors:
        form = forms.Form(CreateNote, data=data, errors=errors)
        notes = await Note.objects.all(user=request.user)
        return templates.TemplateResponse('login.html', {'form': form, 'notes': notes})

    await Note.objects.create(user=request.user, text=validated.text, completed=validated.completed)
    return RedirectResponse(endpoint='homepage')


async def login(request):
    if request.method == 'GET':
        form = forms.Form(Login)
        return templates.TemplateResponse('login.html', {'form': form})

    data = await request.parse()
    validated, errors = await Login().validate_or_error(data=data)
    if errors:
        form = forms.Form(Login, data=data, errors=errors)
        return templates.TemplateResponse('login.html', {'form': form})

    request.session['user'] = user.pk
    return RedirectResponse(endpoint='homepage')


async def logout(request):
    request.session.clear()
    return RedirectResponse(endpoint='login')


async def not_found(request, exc):
    return templates.TemplateResponse("404.html", status_code=404)


async def server_error(request, exc):
    return templates.TemplateResponse("500.html", status_code=500)


# routes.py
routes = [
    Route('/', homepage, name='homepage', methods=['GET', 'POST']),
    Route('/login', login, name='login', methods=['GET', 'POST']),
    Route('/logout', logout, name='logout', methods=['POST']),
    Mount('/static', statics, name='static'),
]


# application.py
events = [
    Event('startup', database.connect),
    Event('shutdown', database.disconnect)
]

middleware = [
    Middleware(TrustedHostMiddleware, allowed_hosts=ALLOWED_HOSTS),
    Middleware(HTTPSRedirectMiddleware, enabled=not DEBUG),
    Middleware(SessionMiddleware, backend=CookieSignedSessions(secret_key=SECRET_KEY)),
    Middleware(AuthMiddleware, backend=DatabaseAuthBackend(model=User)),
]

exception_handlers = {
    404: not_found,
    500: server_error
}

app = Starlette(
    debug=DEBUG,
    routes=routes,
    events=events,
    middleware=middleware,
    exception_handlers=exception_handlers
)

[ghost]

Opinionated framework may help with stability and understand-ability BUT please do not take it too far. I really like what @tomchristie says about independent components, I think we can all agree on the benefits of that. Having clear philosophy on an open-source project will make it or break it. But too much opinion in terms of “how to use” for the end-user may not be the wisest and people will start forcing their projects into a particular pattern without much forethought. Making opinions optional is always best.

E.G. with Django’s “Apps” most people try to divvy up their project, but then run into issues where their models.py files are all interdependent and tightly coupled, views start needing to query models in different apps, etc. So the next thing they try is just putting everything into a single “App” and end up with monolithic files. Then within a single App try their own organization strategies before eventually re-implementing django’s “Apps” and run into the same problems they had in the first place. Eventually they realize they need to go back to django’s multiple apps, but out-source the coupling to the settings, a process essentially missing from the django docs.

So it’s really important when giving opinions on “how to use” that you back it with philosophy and clear guidance, otherwise people waste too much time on “this is how it’s supposed to be done” instead of “what’s the best way to do this”.