Skip to content

Perform console cursor and text calls via interrupt#3113

Merged
kcgen merged 4 commits intomainfrom
kc/write-to-int10-1
Nov 14, 2023
Merged

Perform console cursor and text calls via interrupt#3113
kcgen merged 4 commits intomainfrom
kc/write-to-int10-1

Conversation

@kcgen
Copy link
Copy Markdown
Member

@kcgen kcgen commented Nov 13, 2023

Description

Picking up from the end of the discussion here, this PR adjusts the the emulated console (the CON device) to no longer immediately inject text into the DOS console and instead uses the INT10 interrupt callback just as real DOS programs do.

The calls used are imported from DOSBox Daum, adapted to avoid duplicate code using a constexpr toggle in the INT10 routines.

Manual testing

Tested debug and sanitizer builds with various console programs at various speeds (XT in low-hundreds, early AT in 500 to 700, and faster 386 and 486 at 3000+ cycles).

Slow XT Class (< 100 cycles)

video0035.mp4

Slow AT Class through 386 and 486 (cycles >= 100 through 10000)

video0037.mp4

Checklist

Please tick the items as you have addressed them. Don't remove items; leave the ones that are not applicable unchecked.

I have:

  • followed the project's contributing guidelines and code of conduct.
  • performed a self-review of my code.
  • commented on the particularly hard-to-understand areas of my code.
  • split my work into well-defined, bisectable commits, and I named my commits well.
  • applied the appropriate labels (bug, enhancement, refactoring, documentation, etc.)
  • checked that all my commits can be built.
  • confirmed that my code does not cause performance regressions (e.g., by running the Quake benchmark).
  • added unit tests where applicable to prove the correctness of my code and to avoid future regressions.
  • made corresponding changes to the documentation or the website according to the documentation guidelines.
  • locally verified my website or documentation changes.

@kcgen kcgen added enhancement New feature or enhancement of existing features video Graphics and video related issues labels Nov 13, 2023
@kcgen kcgen self-assigned this Nov 13, 2023
@kcgen kcgen changed the title Perform all console INT10 cursor and text calls via interrupt Perform console cursor and text calls via interrupt Nov 13, 2023
@johnnovak
Copy link
Copy Markdown
Member

johnnovak commented Nov 13, 2023

Nice one @kcgen.

Out of curiosity, I've tested now how the various commands behave on my Pentium MMX 200 downclocked to 125Mhz (or 100 Mhz, can't remember), and the scrolling is much slower on real hardware in general. I need to set the following cycles values to get approximately the same "scrolling speed" in Staging with this patch when executing various commands:

  • DIRcycles 200
  • MEMcycles 120
  • COPY /?cycles 120
  • TYPE <somefilecycles 40 (yes 😅)

Interestingly, the scrolling speed doesn't change on real hardware much even if I disable all caches in the BIOS and use SETMUL to slow things down as much as possible to slow 386 speeds.

Running real MS-DOS inside Staging doesn't fare much better either; the scrolling is a tiny bit slower in general, but still waaaaay too fast compared to even my Pentium MMX box. This indicates that achieving the same text output rate is not really possible because of general emulation inaccuracies in DOSBox.

In any case, I think this is a step in the right direction and it brings us closer to the RealThing(tm) a tiny bit, but it's still waaaay faster and way off.

Does it matter? Not really, I think 😄 And maybe watching the help text of the various commands visibly scroll is a hard sell for many, but yeah, that's what happens on a real PC. Although, replicating the correct scrolling speeds might trigger some nostalgia in a few people, but let's stop here 😄 And I'm pretty sure you get the correct scrolling speeds in PCem, but that's an apples-to-oranges comparison—their approach to emulation is very different.

@kklobe might chime in as well 😄 I've recorded a few videos, but there's not much point in sharing them. You can accurately replicate the real MS-DOS behaviour with the above low cycles settings from 40 to 200, there's not much else to it.

So yeah, probably let's just merge this and conclude that there are limits to emulation accuracy with the DOSBox approach 🤷🏻 It was a good exercise, at least now we understand the whole text output issue better.

Comment thread dosbox.conf Outdated
@kklobe
Copy link
Copy Markdown
Collaborator

kklobe commented Nov 13, 2023

This is much better - even though it's not "authentic" scrolling speed (didn't real DOS utilities and COMMAND.COM use INT 21 calls like 02 and 09 for writing strings? Maybe that incurred additional overhead?). On main if I do a cls then repeated dir, the screen doesn't change at all, it's like an image viewer. With this change, I at least see activity, so I give this a 🥇.

Copy link
Copy Markdown
Collaborator

@weirddan455 weirddan455 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks good as far as I can tell.

The scrolling speed on a real 8088 is much slower than that first video and your 386-486 video looks more like a fast Pentium. Personally, I don't miss waiting 10 seconds for a DIR command to finish scrolling but you guys do you 😛

@kklobe
Copy link
Copy Markdown
Collaborator

kklobe commented Nov 13, 2023

Code looks good as far as I can tell.

The scrolling speed on a real 8088 is much slower than that first video and your 386-486 video looks more like a fast Pentium. Personally, I don't miss waiting 10 seconds for a DIR command to finish scrolling but you guys do you 😛

😁 indeed! If I really, really get nostalgic and just have to have that experience, I can always jump over to https://www.pcjs.org for a quick hit.

@FeralChild64
Copy link
Copy Markdown
Collaborator

On my system scrolling speed is still very fast, unless I go very low with cycles. Even if we issue a real interrupt, most of the DOS/BIOS processing is done by the native code (with native speed), not emulated one.

But the whole processing (using the actual interrupts) is now definitely more realistic.

@kcgen
Copy link
Copy Markdown
Member Author

kcgen commented Nov 13, 2023

The next step would be to emulate video-card memory read and write latency, like DOSBox-X's configurable vmemdelay that touches src/hardwrae/vga_memory.cpp.

I have a feeling DOSBox didn't emulate this because there was never an XT or AT "specification" that bound video memory IO latency to some threshold.

The delay was always seen as undesirable: people writing games wanted faster video handling, and people coding 'printf' in C wanted content printed as fast as possible. Plus.. video card manufacturers constantly pursued faster chips (so the landscape was never fixed).

Game devs learned to draw on vblank.. which works with slow or really fast video cards, and the few that didn't either tear or need to be throttled at the CPU level to pace their game logic.

I scanned all of eXoDOS's conf files (7300+ games), and none are using DOSBox-X's vmemdelay setting, which is at least is some comfort that we may not need this.

Would be very interesting to know if there are any games that rely sole on video card latency to "throttle" their game speed; that if latency goes down, the game breaks or runs too fast.

Will merge this for now - but maybe vmemdelay still makes sense, just for a more nostalgic feel on the CLI :-)

@kcgen
Copy link
Copy Markdown
Member Author

kcgen commented Nov 13, 2023

Even if we issue a real interrupt, most of the DOS/BIOS processing is done by the native code (with native speed), not emulated one.

Yup; really everything 'terminates' in native code in DOSBox, but the interrupts are emulated within the 1ms tick loop, and how much work is accomplished per millisecond is a function of cycles.

(but the emulated video card is still really fast :-)

Interestingly, the scrolling speed doesn't change on real hardware much even if I disable all caches in the BIOS and use SETMUL to slow things down as much as possible to slow 386 speeds.

conclude that there are limits to emulation accuracy with the DOSBox approach

Yup, the physical video card is bottleneck'ing the display rate, even when the CPU is throttled down.

Imported-from: origin/forks/daum-scm-restoration

Signed-off-by: kcgen <kcgen@users.noreply.github.com>
@rderooy
Copy link
Copy Markdown
Collaborator

rderooy commented Nov 13, 2023

The only game that I could quickly find that seems to benefit from vmemdelay is Super Zaxxon
joncampbell123/dosbox-x#4197 (comment)

@johnnovak
Copy link
Copy Markdown
Member

johnnovak commented Nov 13, 2023

I played around with vmemdelay in DOSBox-X because I hoped it would help with the Dungeon Keeper map menu and Quest for Glory II intro scroll speed issues. It had absolutely zero effect.

I'm really curious where these extra latencies are coming from that slow down the scrolling in DK and QfG. I think I'll create issue tickets with video recordings so we at least keep track of them.

In QfG II in the intro sequence sometimes there is full screen scrolling between the ground and the sky. On my fast MMX box you can see the scrolling, but in DOSBox you can't see it it's so fast, unless you use comically low cycles settings that make the game unplayable.

Haven't tried it PCem yet, but that might reveal a few things. Maybe these games are calling some BIOS routines that do specific things that our BIOS emulation don't emulate or too quickly. Well, you can run actual BIOS images in X: joncampbell123/dosbox-x#193 (comment)

But then it might be some quirks of the VGA/VESA BIOS, or maybe something completely different 🤷🏻‍♂️ You can only use the actual VGA BIOS in PCem AFAIK (or maybe not even there?)

It's worthwhile to note that I'm running CompactFlash cards in this MMX machine of mine with CF to IDE adapters. So disk I/O bottlenecks are effectively ruled out at the drive level (but there's the IDE controllers, the BIOS, and the DIR command itself).

Anyway, accurate emulation is a rabbit whole of its own. Approving the PR in the meantime 😄

@kklobe
Copy link
Copy Markdown
Collaborator

kklobe commented Nov 13, 2023

I'm really curious where these extra latencies are coming from that slow down the scrolling in DK and QfG. I think I'll create issue tickets with video recordings so we at least keep track of them.

Sounds like a great excuse to play with Tracy. If someone can make a repro pack I'd be willing to take a chop at it.

Comment thread src/ints/int10_char.cpp Outdated
Copy link
Copy Markdown
Member

@johnnovak johnnovak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice enhancement @kcgen 🥳

@johnnovak
Copy link
Copy Markdown
Member

I'm really curious where these extra latencies are coming from that slow down the scrolling in DK and QfG. I think I'll create issue tickets with video recordings so we at least keep track of them.

Sounds like a great excuse to play with Tracy. If someone can make a repro pack I'd be willing to take a chop at it.

Will do sometime this week and will tag you. Curious what you'll find out!

Replaces the 'Real' function prefixes with a
'ViaInterrupt' postfix to more clearly
indicate that the calls are made via the
interrupt handler (and run just like any other
DOS program is run).
Uses a template enum class to toggle which calls are used
inside the affected functions.

These are wrapped and given standard (non-template) INT10_
module function names for use in other modules.
This avoids instantly injecting text into the DOS console and
instead schedules the INT10 call via the same interrupt callback
as real DOS programs do.
@kcgen kcgen force-pushed the kc/write-to-int10-1 branch from 0852483 to 2888944 Compare November 14, 2023 00:29
@kcgen
Copy link
Copy Markdown
Member Author

kcgen commented Nov 14, 2023

Thanks for the tests and reviews, @johnnovak, @FeralChild64, @kklobe, @weirddan455, and @rderooy.

What started out as something we thought was a minor glitch turned out to reveal that DOSBox's emulation is realistic and robust (despite being unthrottled at the ISA video card level).. and that this might lead to introducing more video-side latency.

@kcgen kcgen merged commit 30c5af0 into main Nov 14, 2023
@kcgen kcgen deleted the kc/write-to-int10-1 branch November 14, 2023 02:55
@johnnovak johnnovak added the DOS Issues related to DOS integration or DOS commands label Dec 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

DOS Issues related to DOS integration or DOS commands enhancement New feature or enhancement of existing features video Graphics and video related issues

Projects

No open projects
Status: Done

Development

Successfully merging this pull request may close these issues.

7 participants