Re: [Freedos-user] Any interest in 486, 586, 686 kernels?

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: [Freedos-user] Any interest in 486, 586, 686 kernels?

Louis Santillan
I apologize for mis-posting to fd-user previously.

On Thu, May 16, 2013 at 10:47 AM, Tom Ehlert <[hidden email]> wrote:
Dear Louis,

a few points

a) the FreeDOS project isn't very interested in a BC5 compiled
kernel because BC5 isn't freely available/open source;
I also doubt the output of BC5 will be significant better then the OW
output.
feel free to experiment, but don't expect us to be excited ;)

I'm not sure how you can say the FreeDOS project isn't interested in a BC5 kernel.  The BC5 makefiles I found in the kernel sources I didn't write.  Bart last worked on them 9 years ago.  Bit rotten for sure and OW became usable in that time.  So yes, priorities change, but I'm taking the posted FreeDOS Roadmap, as goals and stretch goals for the project.  I read (paraphrasing): built-in networking, built-in USB, integrated DPMI, 32-bit & 64-bit support, device driver imports, automated regression testing. I've done a couple of simple tests and I am getting 32-bit register code from my copy of BC5.  The Roadmap is reason enough for me, personally, to continue to 'experiment' as you say.  There's no way of getting 32-bit real mode code from OW.  So for now, until someone teaches OW some new tricks, I'll work with BC5.

 
> So, something in the make files/build files is skipping building a concrete
> GLOBAL for ReturnAnyDosVersionExpected for BC5.  There's a MAIN define
> checked but the build process doesn't seem to get defined anywhere. :/

b) when trying to port the kernel to a new compiler, you should be
able to fix such issues yourself. generate assembler output, see what
is wrong. you will need this as the FreeDOS uses the
 'interesting memory model (TM)'

Again, I'm not doing any porting.  And I do intend to work this issue.  However, software development, like many human endeavors, is best done collaboratively & socially, IMO.  If someone in the last 9 years has compiled the kernel with BC5, they might have tips for me.  Heck, I remember when the kernel was TASM/BC only, and only a select few could afford to contribute.  I advocated back then (almost 15 years ago) for porting to open tools.  I'm glad the early FreeCOM/Kernel developers had made the effort to port to open tools.
 
> Need to do more digging.
c) no need to write 'need more digging' type of mails. use your
twitter account for that.

I don't have a twitter account.  Feel free to filter emails from me to your spambox.

------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d
_______________________________________________
Freedos-kernel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/freedos-kernel
Reply | Threaded
Open this post in threaded view
|

Re: Any interest in 486, 586, 686 kernels?

Eric Auer-3

Hi Louis,

sort of a long response - it seems hard to make a short point here:

> FreeDOS Roadmap, as goals and stretch goals for the project.  I read
> (paraphrasing): built-in networking, built-in USB, integrated DPMI, 32-bit

The fd32 project works or worked on the DPMI aspect. As far as I
remember, the performance gain was minimal, but consider interest
in terms of style to consider a protected mode kernel which has
both DPMI and VM86 visibility. It would be a sort of DOSBOX then.

Regarding built-in networking and USB, I am personally more for
the existing loadable driver model. The BIOS gives you enough
support to reach the point where you can load drivers in config
sys. Even when booting via network (PXE) or from CD/DVD/BD, you
can still use a MEMDISK (bootable ramdisk) to get started...

On the other hand, I think it would be an interesting experiment
to have a kernel which can load files from at least the root dir
of ISO9660 or UDF disks and similar (ext2? ntfs?) and which can
directly interpret GPT partition tables. The latter would have
significant practical use. Not the former: When you boot from
CD/DVD/BD, you do typically want a writeable ramdisk anyway, so
you can just as well boot with the help of a MEMDISK which lets
you load DOS CD/... drivers from there, making drivers for that
as fixed parts of the kernel less interesting. Also, they are
harder to update than separate drivers and you cannot easily
skip them at boot to save DOS memory.

Good question whether you want built-in USB: I would say no, you
already need the BIOS to help to boot from USB. After that, you
want a modular system of USB drivers (unless the BIOS provides
support for all relevant USB devices) to let you access disks,
sticks, mice, keyboards and so on. Also, devices for which there
is no popular DOS or BIOS interface, such as network or sound,
cannot have kernel drivers in a useful way. For networking, the
packet driver interface is just one facet in a complex world, we
could think about some additions to make it easier to write new
drivers or to do common things with the network. For sound, the
lack of popular interfaces means that you basically have to use
protected mode to simulate a popular HARDWARE (SoundBlaster etc)
and then let your actual driver play whatever the soundblaster
would have played given the detected I/O with the simulation.

> & 64-bit support, device driver imports, automated regression testing.

Some automated or semi-automatic quality checks seem interesting.
One could think of code style checks, audits, automated comparison
of how well different artificial test cases or real software runs
work with different kernel (micro-) versions in some VM etcetera.

Regarding 64 bit support, there are some experiments with letting
DPMI drivers use more than 4 GB of RAM. To make better use of it,
you would have to define new interfaces (XMS, EMS, DPMI, VCPI...)
and hope that DOS software would be written which uses them.

Just using 32- or 64-bit calculations (not RAM addresses) is another
story: Probably the kernel can be smaller by using more 32-bit calc
but not really faster. Using memcopy optimized for more modern CPU
(32, 64, 128, ... bit, FPU, MMX, SSE, ...) will also not make a real
difference, as the DOS kernel itself does not move a lot of data.

In short, there are certainly points in which DOS can be extended,
but I personally doubt that many or even any of them should start
in the kernel. Separate drivers and other software fit better.

Maybe we should discuss the roadmap again in the first place?

Regards, Eric



------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d
_______________________________________________
Freedos-kernel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/freedos-kernel
Reply | Threaded
Open this post in threaded view
|

Re: [Freedos-user] Any interest in 486, 586, 686 kernels?

Hans-2
In reply to this post by Louis Santillan
On 20/05/2013 02:11, Louis Santillan wrote:
I apologize for mis-posting to fd-user previously.

On Thu, May 16, 2013 at 10:47 AM, Tom Ehlert <[hidden email]> wrote:
Dear Louis,

a few points

a) the FreeDOS project isn't very interested in a BC5 compiled
kernel because BC5 isn't freely available/open source;
I also doubt the output of BC5 will be significant better then the OW
output.
feel free to experiment, but don't expect us to be excited ;)

I'm not sure how you can say the FreeDOS project isn't interested in a BC5 kernel. 

Tom meant to say "don't expect me to be excited" ;) he is obviously not speaking for everybody on this mailing list. I for one would be interested to see size/speed numbers for any compiler free or commercial.

Good luck with BC5,

Hans
www.ht-lab.com


------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d
_______________________________________________
Freedos-kernel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/freedos-kernel
Reply | Threaded
Open this post in threaded view
|

Re: Any interest in 486, 586, 686 kernels?

Louis Santillan
In reply to this post by Eric Auer-3

On Sun, May 19, 2013 at 7:06 PM, Eric Auer <[hidden email]> wrote:

Hi Louis,

sort of a long response - it seems hard to make a short point here:

> FreeDOS Roadmap, as goals and stretch goals for the project.  I read
> (paraphrasing): built-in networking, built-in USB, integrated DPMI, 32-bit

The fd32 project works or worked on the DPMI aspect. As far as I
remember, the performance gain was minimal, but consider interest
in terms of style to consider a protected mode kernel which has
both DPMI and VM86 visibility. It would be a sort of DOSBOX then.

I would expect performance gain to be minimal.  Maybe there could be Low/HMA/UMB memory savings with a different architecture.  Hard to say.
 
Regarding built-in networking and USB, I am personally more for
the existing loadable driver model. The BIOS gives you enough
support to reach the point where you can load drivers in config
sys. Even when booting via network (PXE) or from CD/DVD/BD, you
can still use a MEMDISK (bootable ramdisk) to get started...

I lean this way too w/respect to drivers.  Built-in's biggest advantage is simplicity in user configuration. However, networking seems to be lacking throughput speed with either mTCP or Watt32 apps.  I'm not sure if it's a packet driver, TCP/IP stack, app (doubt), or kernel (doubt) issue.  My guess is that it's a 16-bit MOV/LD* loops, the lack of zero-copy networking, and the switch between kernel and app.  I've seen some reference to the same issue on USB drivers as well (USB 1.1 speed from USB 2.0 devices or 3.0 devices).  And there seems to be 3 ways to get USB working on DOS: USBDOS, DOSUSB, Panasonic/ASPI Method.  I need to play with those.

 
On the other hand, I think it would be an interesting experiment
to have a kernel which can load files from at least the root dir
of ISO9660 or UDF disks and similar (ext2? ntfs?) and which can
directly interpret GPT partition tables. The latter would have
significant practical use. Not the former: When you boot from
CD/DVD/BD, you do typically want a writeable ramdisk anyway, so
you can just as well boot with the help of a MEMDISK which lets
you load DOS CD/... drivers from there, making drivers for that
as fixed parts of the kernel less interesting. Also, they are
harder to update than separate drivers and you cannot easily
skip them at boot to save DOS memory.

Yes! I agree!  The FD Kernel needs to speak MBR, GPT, ISO9660 & UDF (including El Torito), NTFS, Ext2/3/4 to stay relevant.  Probably HFS+ as well.  Linux and/or FUSE should be helpful here.

 
Good question whether you want built-in USB: I would say no, you
already need the BIOS to help to boot from USB. After that, you
want a modular system of USB drivers (unless the BIOS provides
support for all relevant USB devices) to let you access disks,
sticks, mice, keyboards and so on. Also, devices for which there

I'd want FD to be USB-ready (built-in or driver) for Floppy, Zip/Jaz/SparQ/LS-120 (similar drives), HDD, CD/DVD/BD, mice, gamepads/joysticks, keyboards.

 
is no popular DOS or BIOS interface, such as network or sound,
cannot have kernel drivers in a useful way. For networking, the
packet driver interface is just one facet in a complex world, we
could think about some additions to make it easier to write new
drivers or to do common things with the network. For sound, the
lack of popular interfaces means that you basically have to use
protected mode to simulate a popular HARDWARE (SoundBlaster etc)
and then let your actual driver play whatever the soundblaster
would have played given the detected I/O with the simulation. 
 

I think Jim Hall's suggestion of a virtualized OS is more applicable here.  Something like MOL (Mac On Linux), or QEMU User Space Emulation (ala Wine) with device redirection to generic hardware might be best.
 

> & 64-bit support, device driver imports, automated regression testing.

Some automated or semi-automatic quality checks seem interesting.
One could think of code style checks, audits, automated comparison
of how well different artificial test cases or real software runs
work with different kernel (micro-) versions in some VM etcetera.

Just need people to start writing tests. :D

 

Regarding 64 bit support, there are some experiments with letting
DPMI drivers use more than 4 GB of RAM. To make better use of it,
you would have to define new interfaces (XMS, EMS, DPMI, VCPI...)
and hope that DOS software would be written which uses them.
 
I've seen cwsdpmi r7 supports upto 64GB address space.  Some new interfaces might be nice to program to, but I'd advocate step one to be making use of such extensions transparent to old programs.  I'd only count on BIOS or other specialty hardware/firmware developers to be interested in such new interfaces.
 
Just using 32- or 64-bit calculations (not RAM addresses) is another
story: Probably the kernel can be smaller by using more 32-bit calc
but not really faster. Using memcopy optimized for more modern CPU
(32, 64, 128, ... bit, FPU, MMX, SSE, ...) will also not make a real
difference, as the DOS kernel itself does not move a lot of data.


I think memcopy becomes more important the more we look to implement advanced hardware like USB1.1/2.0/3.0, 10/100/1000 ethernet.  They're all based on memory mapped IO.
 
In short, there are certainly points in which DOS can be extended,
but I personally doubt that many or even any of them should start
in the kernel. Separate drivers and other software fit better.

Maybe we should discuss the roadmap again in the first place?

I'd like to see more discussion.  In the mean time, I'll keep experimenting. 

------------------------------------------------------------------------------
Try New Relic Now & We'll Send You this Cool Shirt
New Relic is the only SaaS-based application performance monitoring service
that delivers powerful full stack analytics. Optimize and monitor your
browser, app, & servers with just a few lines of code. Try New Relic
and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_may
_______________________________________________
Freedos-kernel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/freedos-kernel
Reply | Threaded
Open this post in threaded view
|

Re: [Freedos-user] Any interest in 486, 586, 686 kernels?

tom ehlert
In reply to this post by Louis Santillan
> I'm not sure how you can say the FreeDOS project isn't interested in a BC5
> kernel.
because I was around when the kernel was ported to MSVC, BC5, OW (in
that order)

> The BC5 makefiles I found in the kernel sources I didn't write.
>  Bart last worked on them 9 years ago.
right. an since OW became an *free* option, MSVC and BC were dropped

>    Bit rotten for sure
the last time I checked bits don't rot

>    and OW became
> usable in that time.  So yes, priorities change, but I'm taking the posted
> FreeDOS Roadmap, as goals and stretch goals for the project.  I read
> (paraphrasing): built-in networking, built-in USB, integrated DPMI, 32-bit
> & 64-bit support, device driver imports, automated regression testing.
all great - and so far off reality that is isn't even funny.
is was never even discussed, let alone agreed upon.

> I've
> done a couple of simple tests and I am getting 32-bit register code from my
> copy of BC5.
of course this is a huge step towards 'built in USB, 64 bit, bla'

> The Roadmap is reason enough for me, personally, to continue
> to 'experiment' as you say.  There's no way of getting 32-bit real mode
> code from OW.
OW outputs enough 32-bit real mode code to get the job done.
is the BC5 code smaller/faster in a significant way ?


> Again, I'm not doing any porting.  And I do intend to work this issue.
>  However, software development, like many human endeavors, is best done
> collaboratively & socially, IMO.  If someone in the last 9 years has
> compiled the kernel with BC5, they might have tips for me.  Heck, I
> remember when the kernel was TASM/BC only, and only a select few could
> afford to contribute.  I advocated back then (almost 15 years ago) for
> porting to open tools.  I'm glad the early FreeCOM/Kernel developers had
> made the effort to port to open tools.
it was a pleasure ;)


tom


------------------------------------------------------------------------------
Try New Relic Now & We'll Send You this Cool Shirt
New Relic is the only SaaS-based application performance monitoring service
that delivers powerful full stack analytics. Optimize and monitor your
browser, app, & servers with just a few lines of code. Try New Relic
and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_may
_______________________________________________
Freedos-kernel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/freedos-kernel
Reply | Threaded
Open this post in threaded view
|

Re: Any interest in 486, 586, 686 kernels?

Eric Auer-3
In reply to this post by Louis Santillan

Hi!

Your vision on DOS is somewhat, well, interesting ;-) So
there is a lot to chat about, although I am not sure if
the KERNEL list is the right place for this topic. DPMI:

> I would expect performance gain to be minimal.  Maybe there could be
> Low/HMA/UMB memory savings with a different architecture.  Hard to say.

Well there could, but why would it matter? More heavy DOS
software normally uses DOS extenders / DPMI anyway, which
means it does not care about how much low DOS RAM is free.

The HMA is mostly used for the kernel and buffers, so as
long as the kernel fits in there, no others have a heavy
need for it. Only a few drivers may use it "UMB style".
Also, caches put bigger buffers in XMS, not needing HMA.

Finally UMB is mostly used for drivers: You could write
drivers that use DOS extenders / DPMI if you really have
a lack of space. Actually the simulation of SoundBlaster
16 that comes with "SoundBlaster" PCI works like that.

Probably also some commercial NTFS drivers, because NTFS
is complex and you do not want to spend 100s of kilobytes
of DOS memory only for loading a filesystem driver...

USB, PXE and CD/DVD/BD drivers as drivers / in memdisk:

> I lean this way too w/respect to drivers.  Built-in's biggest advantage is

You still have to configure built-in drivers, you only
avoid the risk that you forget to include the file in
your boot disk ;-)

> simplicity in user configuration. However, networking seems to be lacking
> throughput speed with either mTCP or Watt32 apps.  I'm not sure if it's a
> packet driver, TCP/IP stack, app (doubt), or kernel (doubt) issue.

Networking in DOS means that your app has a compiled-in
network stack which communicates with a packet driver to
get the low-level hardware stuff done. You often have a
small buffer for that and little concurrency. There may
be a bit of IRQ and DMA, but "big" operating systems are
more relaxed about juggling with multiple streams of net
data with support from complex chips on your network card.
Note that this is just an educated guess: Ask our experts!

> guess is that it's a 16-bit MOV/LD* loops, the lack of zero-copy
> networking, and the switch between kernel and app.  I've seen some

No switch: The kernel does not network at all and there is
only one app at a time using the network. Depending on how
your packet driver and network stack library work, you do
not need many steps of copying either and the transfer to
or from a buffer is unlikely to be a big bottleneck with
modern CPU, I think. However, as said, you probably work
with little bits of data and small buffers in DOS, because
you may have less orchestration between stack and driver.

> reference to the same ... USB drivers as well (USB 1.1 speed from USB
> 2.0 devices or 3.0 devices).  And there seems to be 3 ways to get USB

That problem far much more trivial than you might think:
USB 2 and 3 are controlled in ways that differ a lot from
USB 1, so many drivers simply do not talk USB 2 or 3 at
all. This is not like AGP, PCI or PCIe where you flip a
few bits and suddenly I/O to your graphics card is fast.

It is more like the difference between paged graphics at
a000:0 and a linear framebuffer, to stay in the example.
However, there is no linear USB. Talking USB the 2 or 3
way is just a quite different language, but as you know,
at least one shareware driver speaks it and knows the
dialect of at least some relatively widespread chips...

> working on DOS: USBDOS, DOSUSB, Panasonic/ASPI Method. I need to play

Enjoy :-) And maybe do some benchmarks. Even the shareware
driver should work long enough for that - I think it just
blocks after a while after each boot, but is not locked in
terms of how many days or years you have it installed.

>> On the other hand, I think it would be an interesting experiment
>> to have a kernel which can load files from at least the root dir
>> of ISO9660 or UDF disks and similar (ext2? ntfs?) and which can
>> directly interpret GPT partition tables...

> Yes! I agree!  The FD Kernel needs to speak MBR, GPT, ISO9660 & UDF
> (including El Torito), NTFS, Ext2/3/4 to stay relevant. Probably HFS+ as
> well.  Linux and/or FUSE should be helpful here.

Uhm no. You do not agree ;-) Yes the kernel should speak both
MBR and GPT. Something about 4k sector size is also a good
idea. However, El Torito is only so-so as CD/DVD/BD driver
and Jack's drivers are probably better. It would be fun from
an academic point of view to have ElTorito-ISO9660-GPT in a
kernel, but even Linux works great with kernels-without-any-
disk-drivers when you put the drivers in the boot-ramdisk to
load them as separate files from there. DOS with MEMDISK is
basically the same.

Having (separately loaded) drivers for ISO9660 (we do, even
with long file name support) and UDF (only experimental and
abandoned??) is indeed important for DOS. Also, somebody may
want to undust the old EXT2 semi-drivers called LTOOLS, they
probably still have some sort of ancient limitation such as
lacking LBA support or getting confused by non-ancient EXT2
or even EXT3 or EXT4 filesystems. Note that LTOOLS, similar
to SMBCLIENT, works a bit like FTP clients: You type some
command and files get transferred. That is much easier than
making a drive letter, because a Windows or Linux drive is
probably way more complex than can be represented as a good
old DOS drive letter and the drivers would also be way more
complex than the amount of RAM that you want to invest for
a DOS driver (loadable or part of the kernel) would allow.

This also holds for HFS+ and NTFS and ExFAT and so on, of
course. Even if you take the effort to make a super-powered
DOS extended / DPMI driven driver, you would still end up
having your cool Windows or Linux drive with devices and
pipelines and long Unicode names and so on represented as
a boring, maybe even 8.3-named shim for the sake of letting
your 1980 GIF viewer access your Windows 8 clip-arts ;-)

If you access your Windows and Linux drive using some DOS
extended file manager with loadable filesystem modules,
you might get a better user experience with less headache
about DOS RAM than when you try to over-boost DOS itself.

> I'd want FD to be USB-ready (built-in or driver) for Floppy,
> Zip/Jaz/SparQ/LS-120 (similar drives), HDD, CD/DVD/BD, mice,
> gamepads/joysticks, keyboards.

A built-in driver in that sense would be any driver for any
DOS. It does not have to have any relation with the kernel
of FreeDOS in particular... If fancy USB drives such as ZIP
behave well, they should all be covered by the same storage
driver for DOS. But if you read the docs from Bret, you see
that USB storage devices kind of default to misbehave :-p

Similarily, CD/DVD/BD go in one category, as long as your
driver speaks the general protocol (or for non-USB: speaks
your controller such as SATA or IDE/ATA/ATAPI) and as long
as you have a CD-like / network-redirector-like driver for
understanding the ISO9660 or other "optical" filesystem.

I think your contributions would be welcome in writing some
components a la SHSU-UDF (SHSUCDX but then UDF) or modules
for Bret's drivers to understand GPT with all FAT sizes and
maybe maybe the most basic support for the most widespread
"modern" filesystems. Say some NTFS version and EXT2 with a
bit of resistence to EXT3 and EXT4 at the expense of being
only read-only or something... As long as you get a bit in
touch with your Windows and Linux data... I think writing
from DOS to those is a risky idea in the first place. That
would be like using a disk hex editor to fix your DirectX.

Luckily Bret already supports mice, keyboards and game :-)

> I think Jim Hall's suggestion of a virtualized OS is more applicable here.
>  Something like MOL (Mac On Linux), or QEMU User Space Emulation (ala Wine)
> with device redirection to generic hardware might be best.

That would be called dosbox and dosemu then, already exists ;-)
You could contribute by making a TINY Linux distro around them
which mostly serves as workhorse for all sorts of drivers, for
hardware and filesystems, network, showing your CGA game in
suitable form on your holographic tablet touch screen, etc...

>> Some automated or semi-automatic quality checks seem interesting.

> Just need people to start writing tests. :D

Feel invited. Note that with automated code quality checks you
will initially just find out that DOS is convoluted and ugly.
But it has to be like that to be compatible. The hard part is
to find out what is necessarily ugly and what are real bugs :-p

> I've seen cwsdpmi r7 supports upto 64GB address space.  Some new interfaces
> might be nice to program too, but I'd advocate step one to be making use of
> such extensions transparent to old programs.

How are your experiences with CWSDPMI r7? Can you help making
it more stable, at least by giving feedback to the maintainer
who probably does not really have but will enjoy it if people
enjoy the proof of concept? :-)

You are right about the transparency in some aspects: If your
interface can only allocate 100 MB for some reasons, you can
still offer ten programs to allocate 100 MB each as long as
you know that you actually have 1 GB and as long as you can
let the driver keep an overview which 100 MB is of which app.

> I think memcopy becomes more important the more we look to implement
> advanced hardware like USB1.1/2.0/3.0, 10/100/1000 ethernet.  They're
> all based on memory mapped IO.

How do you know? Maybe they can also just DMA to where you
need the data to be in the end? In particular with USB, the
problem seems to be more that USB 1 is not for big chunks
of data and requesting many small chunks has big overhead,
no matter how fast you memcopy them. You get similar issues
with disk I/O, DOS caches rarely pool your I/O to read-ahead
megabytes of your MPEG or combine several FAT changes before
writing them in some smart and safe way. Instead, expect DOS
apps to read your MPEG 32 kB at a time and update the FAT at
a ratio of a few bytes at a time. Combine that with the IOPS
rating of a typical USB2 stick and the overhead of only using
USB1 to talk to it and you see why sorting your family pics
on that stick or some SD card is not very smooth inside DOS.

You could try to write a layer of cleverness and pooling for
some cache, or a complete cache, but note that people might
actually prefer basic: You write it, it is slow, but you can
unplug that USB stick or even your whole PC after a fraction
of a second and the data already is there and consistent...

>> Maybe we should discuss the roadmap again in the first place?

> I'd like to see more discussion.  In the mean time, I'll keep
> experimenting.

Enjoy :-) As Tom said between the lines, the (more long term)
roadmap is more visionary and less based on consensus or some
practical "what do we NEED to add NEXT" considerations. The
roadmap for 1.0 and 1.1 was more like that. Today I would say
the question is more which bits you want to ADD and less what
you want to CHANGE DOS as a whole into. Because after all, it
is quite good that DOS is what it is. It does not need to be
a weak imitation of something else which already is good, too.

Regards, Eric :-)



------------------------------------------------------------------------------
Try New Relic Now & We'll Send You this Cool Shirt
New Relic is the only SaaS-based application performance monitoring service
that delivers powerful full stack analytics. Optimize and monitor your
browser, app, & servers with just a few lines of code. Try New Relic
and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_may
_______________________________________________
Freedos-kernel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/freedos-kernel
Reply | Threaded
Open this post in threaded view
|

Re: [Freedos-user] Any interest in 486, 586, 686 kernels?

Christopher Evans-2
I still think having true functions in batch files would be a good thing turn batch files into more capable language 




On Tuesday, May 21, 2013, Eric Auer wrote:

Hi!

Your vision on DOS is somewhat, well, interesting ;-) So
there is a lot to chat about, although I am not sure if
the KERNEL list is the right place for this topic. DPMI:

> I would expect performance gain to be minimal.  Maybe there could be
> Low/HMA/UMB memory savings with a different architecture.  Hard to say.

Well there could, but why would it matter? More heavy DOS
software normally uses DOS extenders / DPMI anyway, which
means it does not care about how much low DOS RAM is free.

The HMA is mostly used for the kernel and buffers, so as
long as the kernel fits in there, no others have a heavy
need for it. Only a few drivers may use it "UMB style".
Also, caches put bigger buffers in XMS, not needing HMA.

Finally UMB is mostly used for drivers: You could write
drivers that use DOS extenders / DPMI if you really have
a lack of space. Actually the simulation of SoundBlaster
16 that comes with "SoundBlaster" PCI works like that.

Probably also some commercial NTFS drivers, because NTFS
is complex and you do not want to spend 100s of kilobytes
of DOS memory only for loading a filesystem driver...

USB, PXE and CD/DVD/BD drivers as drivers / in memdisk:

> I lean this way too w/respect to drivers.  Built-in's biggest advantage is

You still have to configure built-in drivers, you only
avoid the risk that you forget to include the file in
your boot disk ;-)

> simplicity in user configuration. However, networking seems to be lacking
> throughput speed with either mTCP or Watt32 apps.  I'm not sure if it's a
> packet driver, TCP/IP stack, app (doubt), or kernel (doubt) issue.

Networking in DOS means that your app has a compiled-in
network stack which communicates with a packet driver to
get the low-level hardware stuff done. You often have a
small buffer for that and little concurrency. There may
be a bit of IRQ and DMA, but "big" operating systems are
more relaxed about juggling with multiple streams of net
data with support from complex chips on your network card.
Note that this is just an educated guess: Ask our experts!

> guess is that it's a 16-bit MOV/LD* loops, the lack of zero-copy
> networking, and the switch between kernel and app.  I've seen some

No switch: The kernel does not network at all and there is
only one app at a time using the network. Depending on how
your packet driver and network stack library work, you do
not need many steps of copying either and the transfer to
or from a buffer is unlikely to be a big bottleneck with
modern CPU, I think. However, as said, you probably work
with little bits of data and small buffers in DOS, because
you may have less orchestration between stack and driver.

> reference to the same ... USB drivers as well (USB 1.1 speed from USB
> 2.0 devices or 3.0 devices).  And there seems to be 3 ways to get USB

That problem far much more trivial than you might think:
USB 2 and 3 are controlled in ways that differ a lot from
USB 1, so many drivers simply do not talk USB 2 or 3 at
all. This is not like AGP, PCI or PCIe where you flip a
few bits and suddenly I/O to your graphics card is fast.

It is more like the difference between paged graphics at
a000:0 and a linear framebuffer, to stay in the example.
However, there is no linear USB. Talking USB the 2 or 3
way is just a quite different language, but as you know,
at least one shareware driver speaks it and knows the
dialect of at least some relatively widespread chips...

> working on DOS: USBDOS, DOSUSB, Panasonic/ASPI Method. I need to play

Enjoy :-) And maybe do some benchmarks. Even the shareware
driver should work long enough for that - I think it just
blocks after a while after each boot, but is not locked in
terms of how many days or years you have it installed.

>> On the other hand, I think it would be an interesting experiment
>> to have a kernel which can load files from at least the root dir
>> of ISO9660 or UDF disks and similar (ext2? ntfs?) and which can
>> directly interpret GPT partition tables...

> Yes! I agree!  The FD Kernel needs to speak MBR, GPT, ISO9660 & UDF
> (including El Torito), NTFS, Ext2/3/4 to stay relevant. Probably HFS+ as
> well.  Linux and/or FUSE should be helpful here.

Uhm no. You do not agree ;-) Yes the kernel should speak both
MBR and GPT. Something about 4k sector size is also a good
idea. However, El Torito is only so-so as CD/DVD/BD driver
and Jack's drivers are probably better. It would be fun from
an academic point of view to have ElTorito-ISO9660-GPT in a
kernel, but even Linux works great with kernels-without-any-
disk-drivers when you put the drivers in the boot-ramdisk to
load them as separate files from there. DOS with MEMDISK is
basically the same.

Having (separately loaded) drivers for ISO9660 (we do, even
with long file name support) and UDF (only experimental and
abandoned??) is indeed important for DOS. Also, somebody may
want to undust the old EXT2 semi-drivers called LTOOLS, they
probably still have some sort of ancient limitation such as
lacking LBA support or getting confused by non-ancient EXT2
or even EXT3 or EXT4 filesystems. Note that LTOOLS, similar
to SMBCLIENT, works a bit like FTP clients: You type some
command and files get transferred. That is much easier than
making a drive letter, because a Windows or Linux drive is
probably way more complex than can be represented as a good
old DOS drive letter and the drivers would also be way more
complex than the amount of RAM that you want to invest for
a DOS driver (loadable or part of the kernel) would allow.

This also holds for HFS+ and NTFS and ExFAT and so on, of
course. Even if you take the effort to make a super-powered
DOS extended / DPMI driven driver, you would still end up
having your cool Windows or Linux drive with devices and
pipelines and long Unicode names and so on represented as
a boring, maybe even 8.3-named shim for the sake of letting
your 1980 GIF viewer access your Windows 8 clip-arts ;-)

If you access your Windows and Linux drive using some DOS
extended file manager with loadable filesystem modules,
you might get a better user experience with less headache
about DOS RAM than when you try to over-boost DOS itself.

> I'd want FD to be USB-ready (built-in or driver) for Floppy,
> Zip/Jaz/SparQ/LS-120 (similar drives), HDD, CD/DVD/BD, mice,
> gamepads/joysticks, keyboards.

A built-in driver in that sense would be any driver for any
DOS. It does not have to have any relation with the kernel
of FreeDOS in particular... If fancy USB drives such as ZIP
behave well, they should all be covered by the same storage
driver for DOS. But if you read the docs from Bret, you see
that USB storage devices kind of default to misbehave :-p

Similarily, CD/DVD/BD go in one category, as long as your
driver speaks the general protocol (or for non-USB: speaks
your controller such as SATA or IDE/ATA/ATAPI) and as long
as you have a CD-like / network-redirector-like driver for
understanding the ISO9660 or other "optical" filesystem.

I think your contributions would be welcome in writing some
components a la SHSU-UDF (SHSUCDX but then UDF) or modules
for Bret's drivers to understand GPT with all FAT sizes and
maybe maybe the most basic support for the most widespread
"modern" filesystems. Say some NTFS version and EXT2 with a
bit of resistence to EXT3 and EXT4 at the expense of being
only read-only or something... As long as you get a bit in
touch with your Windows and Linux data... I think writing
from DOS to those is a risky idea in the first place. That
would be like using a disk hex editor to fix your DirectX.

Luckily Bret already supports mice, keyboards and game :-)

> I think Jim Hall's suggestion of a virtualized OS is more applicable here.
>  Something like MOL (Mac On Linux), or QEMU User Space Emulation (ala Wine)
> with device redirection to generic hardware might be best.

That would be called dosbox and dosemu then, already exists ;-)
You could contribute by making a TINY Linux distro around them
which mostly serves as workhorse for all sorts of drivers, for
hardware and filesystems, network, showing your CGA game in
suitable form on your holographic tablet touch screen, etc...

>> Some automated or semi-automatic quality checks seem interesting.

> Just need people to start writing tests. :D

Feel invited. Note that with automated code quality checks you
will initially just find out that DOS is convoluted and ugly.
But it has to be like that to be compatible. The hard part is
to find out what is necessarily ugly and what are real bugs :-p

> I've seen cwsdpmi r7 supports upto 64GB address space.  Some new interfaces
> might be nice to program too, but I'd advocate step one to be making use of
> such extensions transparent to old programs.

How are your experiences with CWSDPMI r7? Can you help making
it more stable, at least by giving feedback to the maintainer
who probably does not really have but will enjoy it if people
enjoy the proof of concept? :-)

You are right about the transparency in some aspects: If your
interface can only allocate 100 MB for some reasons, you can
still offer ten programs to allocate 100 MB each as long as
you know that you actually have 1 GB and as long as you can
let the driver keep an overview which 100 MB is of which app.

> I think memcopy becomes more important the more we look to implement
> advanced hardware like USB1.1/2.0/3.0, 10/100/1000 ethernet.  They're
> all based on memory mapped IO.

How do you know? Maybe they can also just DMA to where you
need the data to be in the end? In particular with USB, the
problem seems to be more that USB 1 is not for big chunks
of data and requesting many small chunks has big overhead,
no matter how fast you memcopy them. You get similar issues
with disk I/O, DOS caches rarely pool your I/O to read-ahead
megabytes of your MPEG or combine several FAT changes before
writing them in some smart and safe way. Instead, expect DOS
apps to read your MPEG 32 kB at a time and update the FAT at
a ratio of a few bytes at a time. Combine that with the IOPS
rating of a typical USB2 stick and the overhead of only using
USB1 to talk to it and you see why sorting your family pics
on that stick or some SD card is not very smooth inside DOS.

You could try to write a layer of cleverness and pooling for
some cache, or a complete cache, but note that people might
actually prefer basic: You write it, it is slow, but you can
unplug that USB stick or even your whole PC after a fraction
of a second and the data already is there and consistent...

>> Maybe we should discuss the roadmap again in the first place?

> I'd like to see more discussion.  In the mean time, I'll keep
> experimenting.

Enjoy :-) As Tom said between the lines, the (more long term)
roadmap is more visionary and less based on consensus or some
practical "what do we NEED to add NEXT" considerations. The
roadmap for 1.0 and 1.1 was more like that. Today I would say
the question is more which bits you want to ADD and less what
you want to CHANGE DOS as a whole into. Because after all, it
is quite good that DOS is what it is. It does not need to be
a weak imitation of something else which already is good, too.

Regards, Eric :-)



------------------------------------------------------------------------------
Try New Relic Now & We'll Send You this Cool Shirt
New Relic is the only SaaS-based application performance monitoring service
that delivers powerful full stack analytics. Optimize and monitor your
browser, app, & servers with just a few lines of code. Try New Relic
and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_may
_______________________________________________
Freedos-kernel mailing list
<a href="javascript:;" onclick="_e(event, &#39;cvml&#39;, &#39;Freedos-kernel@lists.sourceforge.net&#39;)">Freedos-kernel@...
https://lists.sourceforge.net/lists/listinfo/freedos-kernel

------------------------------------------------------------------------------
Try New Relic Now & We'll Send You this Cool Shirt
New Relic is the only SaaS-based application performance monitoring service
that delivers powerful full stack analytics. Optimize and monitor your
browser, app, & servers with just a few lines of code. Try New Relic
and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_may
_______________________________________________
Freedos-kernel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/freedos-kernel
Reply | Threaded
Open this post in threaded view
|

Re: [Freedos-user] Any interest in 486, 586, 686 kernels?

dos386
> Any interest in 486, 586, 686 kernels?

Not really. I don't think that possible 0.01% improvement about
performance or bloat justifies incompatible versions. Note that
8086-compatible EDR-DOS is faster than FreeDOS. Only the mainstream
needs permanently new processors, just to make things slower and more
buggy at the end.

------------------------------------------------------------------------------
Try New Relic Now & We'll Send You this Cool Shirt
New Relic is the only SaaS-based application performance monitoring service
that delivers powerful full stack analytics. Optimize and monitor your
browser, app, & servers with just a few lines of code. Try New Relic
and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_may
_______________________________________________
Freedos-kernel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/freedos-kernel