openwrt/staging/blogic.git
7 years agodrm/nouveau/mmu: implement new vmm frontend
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/mmu: implement new vmm frontend

These are the new priviledged interfaces to the VMM backends, and expose
some functionality that wasn't previously available.

It's now possible to allocate a chunk of address-space (even all of it),
without causing page tables to be allocated up-front, and then map into
it at arbitrary locations.  This is the basic primitive used to support
features such as sparse mapping, or to allow userspace control over its
own address-space, or HMM (where the GPU driver isn't in control of the
address-space layout).

Rather than being tied to a subtle combination of memory object and VMA
properties, arguments that control map flags (ro, kind, etc) are passed
explicitly at map time.

The compatibility hacks to implement the old frontend on top of the new
driver backends have been replaced with something similar to implement
the old frontend's interfaces on top of the new frontend.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/mmu: remove support for old backends
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/mmu: remove support for old backends

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/mmu/gp100,gp10b: implement new vmm backend
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/mmu/gp100,gp10b: implement new vmm backend

Adds support for:
- 64KiB/2MiB big page sizes (128KiB not supported by HW with new PT layout).
- System-memory PTs.
- LPTE "invalid" state.
- (Tegra) Use of video memory aperture.
- Sparse PDEs/PTEs.
- Additional blocklinear kinds.
- 49-bit address-space.

GP100 supports an entirely new 5-level page table layout that provides
an expanded 49-bit address-space.  It also supports the layout present
on previous generations, which we've been making do with until now.

This commit implements support for the new layout, and enables it by
default.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/mmu/gm200,gm20b: implement new vmm backend
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/mmu/gm200,gm20b: implement new vmm backend

Adds support for:
- 64KiB big page size.
- System-memory PTs.
- LPTE "invalid" state.
- (Tegra) Use of video memory aperture.
- Sparse PDEs/PTEs.
- Additional blocklinear kinds.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/mmu/gk104,gk20a: implement new vmm backend
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/mmu/gk104,gk20a: implement new vmm backend

Adds support for:
- 64KiB big page size.
- System-memory PTs.
- LPTE "invalid" state.
- (Tegra) Use of video memory aperture.

Adds support for marking LPTEs invalid, resulting in the corresponding
SPTEs being ignored, which is supposed to speed up TLB invalidates.

On The Tegra side, this will switch to using the video memory aperture
for all mappings.  The HW will still target non-coherent system memory,
but this aperture needs to be selected in order to support compression.

Tegra's instmem backend somewhat cheated to get this effect previously.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/mmu/gf100: implement new vmm backend
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/mmu/gf100: implement new vmm backend

Adds support for:
- 64KiB big page size.
- System-memory PTs.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/mmu/nv50,g84: implement new vmm backend
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/mmu/nv50,g84: implement new vmm backend

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/mmu/nv44: implement new vmm backend
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/mmu/nv44: implement new vmm backend

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/mmu/nv41: implement new vmm backend
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/mmu/nv41: implement new vmm backend

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/mmu/nv04: implement new vmm backend
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/mmu/nv04: implement new vmm backend

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/mmu: implement new vmm backend
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/mmu: implement new vmm backend

This is the common code to support a rework of the VMM backends.

It adds support for more than 2 levels of page table nesting, which
is required to be able to support GP100's MMU layout.

Sparse mappings (that don't cause MMU faults when accessed) are now
supported, where the backend provides it.

Dual-PT handling had to become more sophisticated to support sparse,
but this also allows us to support an optimisation the MMU provides
on GK104 and newer.

Certain operations can now be combined into a single page tree walk
to avoid some overhead, but also enables optimsations like skipping
PTE unmap writes when the PT will be destroyed anyway.

The old backend has been hacked up to forward requests onto the new
backend, if present, so that it's possible to bisect between issues
in the backend changes vs the upcoming frontend changes.

Until the new frontend has been merged, new backends will leak BAR2
page tables on module unload.  This is expected, and it's not worth
the effort of hacking around this as it doesn't effect runtime.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/fb/gm200: enable NV_PFB_MMU_CTRL_USE_FULL_COMP_TAG_LINE where appropriate
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/fb/gm200: enable NV_PFB_MMU_CTRL_USE_FULL_COMP_TAG_LINE where appropriate

To avoid wasting compression tags when using 64KiB pages, we need to
enable this so we can select between upper/lower comptagline in PTEs.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/ltc/gm200: limit NV_MMU_PTE_COMPTAGLINE bits to 16 where required
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/ltc/gm200: limit NV_MMU_PTE_COMPTAGLINE bits to 16 where required

If NV_PFB_MMU_CTRL_USE_FULL_COMP_TAG_LINE is TRUE, then the last bit of
NV_MMU_PTE_COMPTAGLINE is re-purposed to select the upper/lower half of
a compression tag when using 64KiB big pages.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/fifo/nv04-nv40: fix missing nvkm_kmap() calls around ramfc access
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/fifo/nv04-nv40: fix missing nvkm_kmap() calls around ramfc access

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/mmu: handle instance block setup
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/mmu: handle instance block setup

We previously required each VMM user to allocate their own page directory
and fill in the instance block themselves.

It makes more sense to handle this in a common location.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/mmu: remove old vm creation hooks
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/mmu: remove old vm creation hooks

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/mmu/gp100,gp10b: implement vmm on top of new base
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/mmu/gp100,gp10b: implement vmm on top of new base

Adds support for:
- Selection of old/new-style page table layout (GP100MmuLayout=0/1).
- System-memory PDs.

New layout disabled by default for the moment, as we don't have a
backend that can handle it yet.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/mmu/gm200,gm20b: implement vmm on top of new base
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/mmu/gm200,gm20b: implement vmm on top of new base

Adds support for:
- Per-VMM selection of big page size.
- System-memory PDs.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/mmu/gk104,gk20a: implement vmm on top of new base
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/mmu/gk104,gk20a: implement vmm on top of new base

Adds support for:
- Selection of a 64KiB big page size (NvFbBigPage=16).
- System-memory PDs.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/mmu/gf100: implement vmm on top of new base
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/mmu/gf100: implement vmm on top of new base

Adds support for:
- Selection of a 64KiB big page size (NvFbBigPage=16).
- System-memory PDs.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/mmu/nv50,g84: implement vmm on top of new base
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/mmu/nv50,g84: implement vmm on top of new base

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/mmu/nv44: implement vmm on top of new base
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/mmu/nv44: implement vmm on top of new base

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/mmu/nv41: implement vmm on top of new base
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/mmu/nv41: implement vmm on top of new base

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/mmu/nv04: implement vmm on top of new base
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/mmu/nv04: implement vmm on top of new base

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/mmu: implement base for new vm management
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/mmu: implement base for new vm management

This is the first chunk of the new VMM code that provides the structures
needed to describe a GPU virtual address-space layout, as well as common
interfaces to handle VMM creation, and connecting instances to a VMM.

The constructor now allocates the PD itself, rather than having the user
handle that manually.  This won't/can't be used until after all backends
have been ported to these interfaces, so a little bit of memory will be
wasted on Fermi and newer for a couple of commits in the series.

Compatibility has been hacked into the old code to allow each GPU backend
to be ported individually.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/mmu: implement page table sub-allocation
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/mmu: implement page table sub-allocation

GP100 "big" (which is a funny name, when it supports "even bigger") page
tables are small enough that we want to be able to suballocate them from
a larger block of memory.

This builds on the previous page table cache interfaces so that the VMM
code doesn't need to know the difference.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/mmu: implement page table cache
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/mmu: implement page table cache

Builds up and maintains a small cache of each page table size in order
to reduce the frequency of expensive allocations, particularly in the
pathological case where an address range ping-pongs between allocated
and free.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/mmu: automatically handle "un-bootstrapping" of vmm
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/mmu: automatically handle "un-bootstrapping" of vmm

Removes the need to expose internals outside of MMU, and GP100 is both
different, and a lot harder to deal with.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/mmu/gp10b: fork from gf100
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/mmu/gp10b: fork from gf100

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/mmu/gp100: fork from gf100
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/mmu/gp100: fork from gf100

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/mmu/gm20b: fork from gf100
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/mmu/gm20b: fork from gf100

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/mmu/gm200: fork from gf100
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/mmu/gm200: fork from gf100

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/mmu/gk20a: fork from gf100
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/mmu/gk20a: fork from gf100

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/mmu/gk104: fork from gf100
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/mmu/gk104: fork from gf100

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/mmu/g84: fork from nv50
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/mmu/g84: fork from nv50

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/fb/ram: remove old allocators
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/fb/ram: remove old allocators

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau: allocate vram with nvkm_ram_get()
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau: allocate vram with nvkm_ram_get()

This will cause a subtle behaviour change on GPUs that are in mixed-memory
configurations in that VRAM in the degraded section of VRAM will no longer
be used for TTM buffer objects.

That section of VRAM is not meant to be used for displayable/compressed
surfaces, and we have no reliable way with the current interfaces to be
able to make that decision properly.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau: directly handle comptag allocation
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau: directly handle comptag allocation

Another transition step to allow finer-grained patches transitioning to
new MMU backends.

Old backends will continue operate as before (accessing nvkm_mem::tag),
and new backends will get a reference to the tags allocated here.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau: wrap nvkm_mem objects in nvkm_memory interfaces
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau: wrap nvkm_mem objects in nvkm_memory interfaces

This is a transition step, to enable finer-grained commits while
transitioning to new MMU interfaces.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/ltc/gf100-: allocate tagram with nvkm_ram_get()
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/ltc/gf100-: allocate tagram with nvkm_ram_get()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/imem/nv50: allocate memory with nvkm_ram_get()
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/imem/nv50: allocate memory with nvkm_ram_get()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/fb/ram/gt215: allocate training buffer with nvkm_ram_get()
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/fb/ram/gt215: allocate training buffer with nvkm_ram_get()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/fb/ram: add interface to allocate vram as an nvkm_memory object
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/fb/ram: add interface to allocate vram as an nvkm_memory object

Upcoming MMU changes use nvkm_memory as its basic representation of memory,
so we need to be able to allocate VRAM like this.

The code is basically identical to the current chipset-specific allocators,
minus support for compression tags (which will be handled elsewhere anyway).

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/core/memory: add some useful accessor macros
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/core/memory: add some useful accessor macros

Adds support for 64-bit writes, and optimised filling of buffers with
fixed 32/64-bit values.

These will all be used by the upcoming MMU changes.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/core/memory: add reference counting
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/core/memory: add reference counting

We need to be able to prevent memory from being freed while it's still
mapped in a GPU's address-space.

Will be used by upcoming MMU changes.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/core/memory: add mechanism to retrieve allocation granularity
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/core/memory: add mechanism to retrieve allocation granularity

Needed by VMM code to determine whether an allocation is compatible with
a given page size (ie. you can't map 4KiB system memory pages into 64KiB
GPU pages).

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/core/memory: change map interface to support upcoming mmu changes
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/core/memory: change map interface to support upcoming mmu changes

Map flags (access, kind, etc) are currently defined in either the VMA,
or the memory object, which turns out to not be ideal for things like
suballocated buffers, etc.

These will become per-map flags instead, so we need to support passing
these arguments in nvkm_memory_map().

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/core/memory: comptag allocation
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/core/memory: comptag allocation

nvkm_memory is going to be used by the upcoming mmu rework for the basic
representation of a memory allocation, as such, this commit adds support
for comptag allocation to nvkm_memory.

This is very simple for now, in that it requires comptags for the entire
memory allocation even if only certain ranges are compressed.

Support for tracking ranges will be added at a later date.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/ltc: init comptag mm in fb subdev
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/ltc: init comptag mm in fb subdev

A single location for the MM allows us to share allocation logic.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/fb/gf100: clear comptags at allocation time rather than mmu map
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/fb/gf100: clear comptags at allocation time rather than mmu map

We probably don't want to destroy compression data when doing multiple
mappings of a memory object.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/fb: move comptag init out of ram submodule
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/fb: move comptag init out of ram submodule

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/fb: move comptags mm into nvkm_fb
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/fb: move comptags mm into nvkm_fb

We're moving towards having a central place to handle comptag allocation,
and as some GPUs don't have a ram submodule (ie. Tegra), we need to move
the mm somewhere else.

It probably never belonged in ram anyways.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/core/mm: introduce functions to access info about a given allocation
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/core/mm: introduce functions to access info about a given allocation

These will be used in upcoming patches.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/core/mm: have users explicitly define heap identifiers
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/core/mm: have users explicitly define heap identifiers

Different sections of VRAM may have different properties (ie. can't be used
for compression/display, can't be mapped, etc).

We currently already support this, but it's a bit magic.  This change makes
it more obvious where we're allocating from.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau: separate constant-va tracking from nvkm vma structure
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau: separate constant-va tracking from nvkm vma structure

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau: separate buffer object backing memory from nvkm structures
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau: separate buffer object backing memory from nvkm structures

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau: hang drm client of a master
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau: hang drm client of a master

TTM memory allocations will be hanging off the DRM's client, but the
locking needed to do so gets really tricky with all the other use of
the DRM's object tree.

To solve this, we make the normal DRM client a child of a new master,
where the memory allocations will be done from instead.

This also solves a potential race with client creation.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau: consolidate identical functions in nouveau_ttm.c
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau: consolidate identical functions in nouveau_ttm.c

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau: remove unnecessary use of ttm_mem_type_manager::priv
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau: remove unnecessary use of ttm_mem_type_manager::priv

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau: swap loop order in move_notify() hook
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau: swap loop order in move_notify() hook

The conditional is the same for every mapping.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau: simplify const-va map condition
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau: simplify const-va map condition

We don't really care about where the memory is, just that it's compatible
with a VMA allocated for a given page size.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau: split various bo flags out into their own members
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau: split various bo flags out into their own members

It's far more convenient to deal with like this.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau: remove unused sysmem fence code
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau: remove unused sysmem fence code

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau: store nouveau_drm in nouveau_cli, as opposed to drm_device
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau: store nouveau_drm in nouveau_cli, as opposed to drm_device

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/gr/gf100-gk208: copy big page size setting from fb
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/gr/gf100-gk208: copy big page size setting from fb

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/gr/gf100-gk208: make use of init_gpc_mmu() hook to share setup
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/gr/gf100-gk208: make use of init_gpc_mmu() hook to share setup

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/fb: finalise big page size selection in constructor
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/fb: finalise big page size selection in constructor

MMU will need to know this during its constructor, so we can't delay
deciding this until init-time.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/mmu/nv04-nv4x: move global vmm to nvkm_mmu
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/mmu/nv04-nv4x: move global vmm to nvkm_mmu

In a future commit, this will be constructed by common code.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/imem: use fast-path for resume restore
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/imem: use fast-path for resume restore

Before: "imem: init completed in 299277us"
 After: "imem: init completed in  11574us"

Suspend from Fedora 26 gnome desktop on GP102.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/imem: use fast-path for suspend backup
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/imem: use fast-path for suspend backup

Before: "imem: suspend completed in 5540487us"
 After: "imem: suspend completed in 1871526us"

Suspend from Fedora 26 gnome desktop on GP102.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/imem: separate pre-BAR2-bootstrap objects from the rest
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/imem: separate pre-BAR2-bootstrap objects from the rest

These will require slow-path access during suspend/resume.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/imem: switch to kvmalloc/kvfree for suspend/resume backup
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/imem: switch to kvmalloc/kvfree for suspend/resume backup

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/imem: separate suspend/resume backup handling into their own functions
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/imem: separate suspend/resume backup handling into their own functions

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/imem: remove now-unused wrapper for backend objects
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/imem: remove now-unused wrapper for backend objects

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/imem/nv50: support eviction of BAR2 mappings
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/imem/nv50: support eviction of BAR2 mappings

A good deal of the structures we map into here aren't accessed very often
at all, and Fedora 26 has exposed an issue where after creating a heap of
channels, BAR2 space would run out, and we'd need to make use of the slow
path while accessing important structures like page tables.

This implements an LRU on BAR2 space, which allows eviction of mappings
that aren't currently needed, to make space for other objects.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/imem/nv50: prevent fast-path for mapped objects when BAR isn't ready
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/imem/nv50: prevent fast-path for mapped objects when BAR isn't ready

Another piece of solving the "GP100 BAR2 VMM bootstrap" puzzle.

Without doing this, we'd attempt to write PDEs for the lower page table
levels through BAR2 before BAR2 access has been fully initialised.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/imem/nv50: map bar2 write-combined
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/imem/nv50: map bar2 write-combined

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/imem/nv50: embed nvkm_instobj directly into nv04_instobj
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/imem/nv50: embed nvkm_instobj directly into nv04_instobj

This is not as simple as it was for earlier GPUs, due to the need to swap
accessor functions depending on whether BAR2 is usable or not.

We were previously protected by nvkm_instobj's accessor functions keeping
an object mapped permanently, with some unclear magic that managed to hit
the slow-path where needed even if an object was marked as mapped.

That's been replaced here by reference counting maps (some objects, like
page tables can be accessed concurrently), and swapping the functions as
necessary.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/imem/nv50: move slow-path locking into rd/wr functions
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/imem/nv50: move slow-path locking into rd/wr functions

This is to simplify upcoming changes.  The slow-path is something that
currently occurs during bootstrap of the BAR2 VMM, while backing up an
object during suspend/resume, or when BAR2 address space runs out.

The latter is a real problem that can happen at runtime, and occurs in
Fedora 26 already (due to some change that causes a lot of channels to
be created at login), so ideally we'd prefer not to make it any slower.

We'd also like suspend/resume speed to not suffer.

Upcoming commits will solve those problems in a better way, making the
extra overhead of moving the locking here a non-issue.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/imem/nv50: split object map out from api functions
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/imem/nv50: split object map out from api functions

acquire()/boot() will need different logic in addition to performing
the actual mapping.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/imem/nv40: map bar2 write-combined
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/imem/nv40: map bar2 write-combined

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/imem/nv40: embed nvkm_instobj directly into nv04_instobj
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/imem/nv40: embed nvkm_instobj directly into nv04_instobj

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/imem/nv04: directly embed nvkm_instobj into nv04_instobj
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/imem/nv04: directly embed nvkm_instobj into nv04_instobj

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/imem: allow nvkm_instobj to be directly embedded in backend object
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/imem: allow nvkm_instobj to be directly embedded in backend object

This will eliminate a step through the call chain, and give backends
more flexibility.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/core/memory: split info pointers from accessor pointers
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/core/memory: split info pointers from accessor pointers

The accessor functions can change as a result of acquire()/release() calls,
and are protected by any refcounting done there.

Other functions must remain constant, as they can be called any time.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/imem: add some useful debug output
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/imem: add some useful debug output

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/bar/gm107-: wait for instance block binding to complete
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/bar/gm107-: wait for instance block binding to complete

Discovered by accident while working to use BAR2 access to instmem objects
on more paths.

We've apparently been relying on luck up until now!

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/bar: initialise bar2 during oneinit
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/bar: initialise bar2 during oneinit

If we initialise BAR2 earlier, we're able to complete BAR1 setup using
the instmem fast-path.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/bar: prevent BAR2 mapping of objects during destructor
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/bar: prevent BAR2 mapping of objects during destructor

GP100's page table nests a lot more deeply than the GF100-compatible
layout we're currently using, which means our hackish-but-simple way
of dealing with BAR2 VMM teardown won't work anymore.

In order to sanely handle the chicken-and-egg (BAR2's PTs get mapped
into themselves) problem, we need prevent page tables getting mapped
back into BAR2 during the destruction of its VMM.

To do this, we simply key off the state that's now maintained by the
BAR2 init/fini functions.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/bar: modify interface to bar2 vmm mapping
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/bar: modify interface to bar2 vmm mapping

Match API with the BAR1 version.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/bar: modify interface to bar1 vmm mapping
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/bar: modify interface to bar1 vmm mapping

Upcoming changes will remove the nvkm_vmm pointer from nvkm_vma, instead
requiring it to be explicitly specified on each operation.

It's not currently possible to get this information for BAR1 mappings,
so let's fix that ahead of time.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/bar: expose interface to bar2 teardown
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/bar: expose interface to bar2 teardown

Will prevent spurious MMU fault interrupts if something decides to touch
BAR1 after we've unloaded the driver.

Exposed external to BAR so that INSTMEM can use it to better control the
suspend/resume fast-path access.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/bar: expose interface to bar2 initialisation
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/bar: expose interface to bar2 initialisation

If we want to be able to hit the instmem fast-path in a few trickier cases,
we need to be more flexible with when we can initialise BAR2 access.

There's probably a decent case to be made for merging BAR/INSTMEM into BUS,
but that's something to ponder another day.

Flushes have been added after the write to bind the instance block,
as later commits will reveal the need for them.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/bar: implement bar1 teardown
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/bar: implement bar1 teardown

Will prevent spurious MMU fault interrupts if something decides to touch
BAR1 after we've unloaded the driver.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/bar: move bar1 initialisation into its own function
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/bar: move bar1 initialisation into its own function

BAR2 being done for practical reasons, this is just for consistency.

Flushes have been added after the write to bind the instance block,
as later commits will reveal the need for them.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/bar: swap oneinit/init ordering, and rename bar3 to bar2
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/bar: swap oneinit/init ordering, and rename bar3 to bar2

NVIDIA call it BAR2, Linux APIs treat it as BAR3 due to BAR1 being a
64-bit BAR, which I presume take two slots or something.

No actual code changes here, just to make future commits less messy.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/bar: remove NV_PMC_ENABLE_PFIFO twiddling
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/bar: remove NV_PMC_ENABLE_PFIFO twiddling

It's handled by FIFO preinit() now.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/bar/nv50,g84: drop mmu invalidate
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/bar/nv50,g84: drop mmu invalidate

Will already be done by MMU as a result of the PT writes that occur
during BAR2 bootstrapping.

This is likely just a left-over from the days when it was hardcoded.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/fifo: perform reset from preinit
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/fifo: perform reset from preinit

RM appears to do this really early in its initialisation, before DEVINIT.

We currently do this before BAR2 initialisation for some reason.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
7 years agodrm/nouveau/disp: add missing newline in ior debug messages
Ben Skeggs [Tue, 31 Oct 2017 17:56:19 +0000 (03:56 +1000)]
drm/nouveau/disp: add missing newline in ior debug messages

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>