Patching Remote Desktop

I'll start by looking at what's in the directory.

Good, tooling installed. Let me get the PE structure and imports.

Neither binary has an embedded Authenticode signature (SECURITY dir is empty) — they're catalog-signed. Let me confirm and build an analysis harness.

This is a D3D11 renderer. Let me look at the trace strings which likely contain function names.

Let me search for all D3D11 sampler-state creation sites and their filter values.

Now let me tackle bug 2 — the fullscreen→window growth. Let me find the window sizing code.

Found the fullscreen enter path. Let me look at the save/restore-rect helper.

Now let me examine mstsc.exe's window sizing (it computes borders manually via GetSystemMetrics).

Found the container fullscreen enter/leave in mstsc.exe. Let me dump the leave path.

This looks very promising. Let me examine the helper at 0x190a0.

Let me verify by examining the enter-fullscreen path and the other callers of that AdjustWindowRectEx helper.

Confirmed the mechanism. Let me check the third caller and the dispatcher to be sure there's no second growth path.

Now back to bug 1. Let me examine the GDI presenter used by the classic client.

mstsc.exe imports StretchBlt but not SetStretchBltMode. Let me look at that.

Let me trace the DX presenter implementation.

Let me verify my hypothesis by decoding the embedded shaders' sampler usage.

Decisive finding. Let me confirm how many places bind pixel-shader samplers.

I've found both root causes. Let me verify a couple of practical details before patching.

Both root causes are confirmed. Let me check signature status, then patch.

Both patch sites confirmed byte-for-byte. Applying the patches:

Minimal diffs — only the intended instructions plus the PE checksum field. Let me verify the patched code:

Let me smoke-test that the patched binaries load, and confirm the local DLL is the one used.

Both bugs found and patched. Originals are backed up as mstsc.exe.orig / mstscax.dll.orig.

Bug 2 — window grows on every fullscreen→windowed transition

mstsc.exe implements container-handled fullscreen (mstscax.dll calls it through a vtable thunk at RVA 0x1a3e0). In LeaveFullScreen (RVA 0x161a0):

SetWindowPlacement(hwnd, &savedPlacement)   // restores the pre-fullscreen WINDOW rect
GetWindowRect(hwnd, &rc)                    // rc = that window rect (frame included)
SetWindowLong(GWL_STYLE, style | WS_CAPTION|WS_THICKFRAME|WS_MAXIMIZEBOX)
AdjustWindowRectEx(&rc, style, FALSE, exStyle)   // <-- adds the frame a SECOND time
SetWindowPos(hwnd, ..., rc.width, rc.height, SWP_NOMOVE|SWP_FRAMECHANGED)

EnterFullScreen (0x15a8c) saves a raw GetWindowPlacement, and the clamp just above compares rcNormalPosition against [this+0xb0/0xb4], which 0x104b0 computes as window sizes — so the saved rect is unambiguously a window rect. Running AdjustWindowRectEx on it inflates by one non-client frame per cycle: +2×SM_CXSIZEFRAME wide (~16px) and +SM_CYCAPTION+2×frame tall (~39px). That's your "slightly wider, mostly taller."

Patch: mstsc.exe RVA 0x16385 (file offset 0x15785), e8 16 2d 00 00 → 5× 0x90. The restored size is now exactly the saved one. The call's return value was already discarded, so NOPping it has no other effect.

Bug 1 — texture filtering intermittently disabled

mstscax.dll contains exactly one ID3D11Device::CreateSamplerState region, at RVA 0x50b290 (the NV12/YUV→RGB converter used by the hardware H.264/AVC path). It builds a D3D11_SAMPLER_DESC, creates a LINEAR sampler into one stack slot, flips Filter to 0 and creates a POINT sampler into another — then calls PSSetSamplers(0, 1, &point) and never touches the linear one again (it's leaked). Classic wrong-variable bug.

Why it matters: I decoded all 17 embedded DXBC shaders. Every shader in that converter uses ld (texel fetch), which ignores samplers entirely — so the POINT binding does nothing useful there. The only shader in the whole DLL that declares a sampler and uses sample is at RVA 0x773bf0, loaded by 0x511bc8, which is called from the presenter (0x41d6cc / 0x41df24 — the code that D3D11CreateDevices with BGRA_SUPPORT|VIDEO_SUPPORT and scales the desktop into the window). The presenter never creates a sampler of its own, so it runs on whatever is bound to PS slot 0 of the shared context.

That explains every symptom: with no H.264 the slot is empty and D3D11's default sampler applies — MIN_MAG_MIP_LINEAR + CLAMP + MaxLOD=FLT_MAX, which is byte-for-byte the descriptor they built for the unused linear sampler — so scaling is smooth. Once the AVC path initialises (codec is negotiated per connection), POINT is bound on the immediate context and stays there for the life of the process → nearest-neighbour downscale, aliased text, "until you restart it."

Patch: mstscax.dll RVA 0x50b3ea (file offset 0x50a7ea), one byte d7 → df, changing mov rax,[rbp-0x29] to mov rax,[rbp-0x21] so PSSetSamplers binds the linear sampler.

Signing

There was nothing to strip — both files are catalog-signed, not Authenticode-signed; the IMAGE_DIRECTORY_ENTRY_SECURITY directory was already empty (VA 0, size 0) in both. They verified as Valid / Catalog before and now report NotSigned, which is just the catalog hash no longer matching. Nothing enforces that for a user-launched copy. I recomputed the PE checksums; the only bytes that differ from the originals are the checksum field at 0x148 and the patched instructions.

Verification so far

Patched mstsc.exe launches and loads C:\Example\rdp\mstscax.dll (the local copy, not System32's) — confirmed via the process module list.

Two caveats worth stating plainly: I could only verify statically, since testing needs a live RDP session. Bug 2 is fully traced end-to-end and I'm confident. Bug 1's mechanism is inferred from the shader/sampler analysis rather than observed — the patch is safe either way (the converter's own shaders can't be affected, since they don't sample), but if graininess still appears after a connection that uses H.264, tell me and I'll dig into the presenter's own state-setting path.

To revert: copy mstsc.exe.orig mstsc.exe / copy mstscax.dll.orig mstscax.dll.

Found it. The difference is a single mapped file: mstsc.exe.mun.

All checksums match — the current .mun files are compatible with your July build. Now the fix.

Fixed, and verified with a before/after screenshot of your actual patched copy.

What was missing

Not a DLL — mstsc.exe loaded the identical 88 modules in both locations. It's a .mun file.

Since Windows 10 1903, Microsoft moved language-neutral resources (icons, bitmaps, dialog templates) out of system binaries into C:\Windows\SystemResources\<name>.mun. That's why mstsc.exe and mstscax.dll have literally zero icon/bitmap resources of their own — only version info, manifest and typelib.

I found it by enumerating each process's mapped files rather than its loaded modules, since .mun/.mui are opened with LOAD_LIBRARY_AS_DATAFILE and never appear in the module list. The system32 run mapped exactly one file the local run didn't: C:\Windows\SystemResources\mstsc.exe.mun.

Without it you lose the window/taskbar icon, the "Remote Desktop Connection" banner, and the "Show Options" expander — everything else still works because the text comes from the en-US\*.mui files you already copied.

The awkward part

The resource loader resolves the path as:

<folder containing the binary>\..\SystemResources\<filename>.mun

For C:\Windows\System32\mstsc.exe that lands on C:\Windows\SystemResources. For your copy it resolves one level above the binaries, so the files can't go in rdp\ itself. I confirmed this empirically — rdp\SystemResources\ does nothing; the parent works.

So I created:

C:\Example\SystemResources\
    mstsc.exe.mun      (2,084,864 bytes, 10.0.19041.7548)
    mstscax.dll.mun    (1,121,792 bytes, 10.0.19041.7663)
    README.txt         (explains why the folder is there)

I put a README in it because a bare SystemResources folder sitting among your nine project folders is otherwise inexplicable. If you'd rather keep it self-contained, move the binaries into rdp\bin\ (with bin\en-US\) and the SystemResources folder into rdp\ — same relationship, one less stray folder in the parent. Your shortcut's TargetPath would need updating; I left it alone for now.

Version compatibility

Worth flagging: Windows Update replaced your system32 copies on Aug 14 (your originals are the Jul 15 build), and mstscax.dll.mun is from that newer build. The MUN/MUI loader validates a checksum pair stored in each file's MUI resource, so I checked all nine files — base binaries, .muns and .muis all carry identical service checksums and content checksums per component. The newer mstscax.dll.mun is a correct match for your July mstscax.dll. (mstsc.exe.mun wasn't touched by the update anyway.)

.mun files are language-neutral — there's no en-US variant to worry about. The localized half is the en-US\*.mui pair you already have.

I couldn't exercise mstscax.dll.mun without a live session; it holds the in-session UI resources (connection bar, disconnect dialogs), so if anything there still looks bare, tell me and I'll trace it the same way.