Thanks for the tip, that page helped a lot. I'm getting a lot closer to something that works.
I switched to the html provided by that link, but had errors with my code not being signed. To fix that I had to use the pre-signed lwjgl_util_applet provided in the slick download.
Next I got errors due to the logo and progress bar not being loaded. To fix that, I added the lines
<param name="al_logo" value="appletlogo.png">
<param name="al_progressbar" value="appletprogress.gif">
Keep in mind we don't need custom images for these, they're packed up in a jar somewhere.
Lastly the natives.jars for the OSes provided by slick have different name than the one in the tutorial (natives-windows.jar as opposed to windows_native.jar) so be sure to change the name in the html. Kinda silly, but that took me a few minutes to notice.
And now my applet it loading, but after a certain amount of progress I get the error:
An error occured while loading theh applet.
Please contact support to resolve the issue.
Fatal error occured (5): Unable to validate certificate chain. Native entry did not have a certificate chain at all.
Of course the interesting part is that last line. I've heard that all the jars in an applet need to have the same signature or it'll throw errors, so maybe this is the cause. I think you also need to pay Oracle or something to be able to sign, but I've heard of people on this forum getting applets to work without needing signatures, so not really sure what's going on there.
Edit: So now I'm in the process of trying to sign my jar file. Will I need to sign any other jars as well? Or just the jar that contains my game? My game doesn't use any special jars other than the default one.