Slick Forums

Discuss the Slick 2D Library
It is currently Sun May 26, 2013 5:24 am

All times are UTC




Post new topic Reply to topic  [ 16 posts ]  Go to page 1, 2  Next
Author Message
PostPosted: Mon Feb 06, 2012 6:14 pm 
Offline
Slick Zombie

Joined: Sat Jan 27, 2007 7:10 pm
Posts: 1469
I mentioned this in the contribution depot, but I figured it's serious enough that it could do with its own topic.

The problem is: If you create a new empty Image, and then use getGraphics, two OpenGL Textures will be created, but Image.getTexture will only point to one of them (the FBO/pbuffer texture). Therefore, even if you are properly calling destroy() on your Image, a game that creates many offscreen images will eventually run out of memory and crash.

The reason FBO/Pbuffer does not release the old texture is because the user may still want to use it. For example, to add 'decals' to a texture, you might do something like:
Code:
       tile = new Image("res/tex/tile.png"); //original tile
       tile2 = new Image(tile.getTexture()); //tile with 'decal'
       Graphics g = tile2.getGraphics();
       g.setColor(Color.black);
       g.fillOval(5, 5, 10, 10); //draw 'decal'
       g.flush();


In this situation, it wouldn't make sense for getGraphics to release the old texture. But often getGraphics is used for the sake of double buffering or rendering something to an empty texture (e.g. a complex tile map).

Workarounds:
The simplest workaround is to grab the old texture before getGraphics is called, and then release() the old texture after getGraphics. Two textures are still generated, however.

A cleaner workaround is to use something like a NullTexture, so that you are only creating one texture in the end. You would use the class like so:
Code:
tile = new Image(new NullTexture(250, 250));
Graphics g = tile.getGraphics();
//... render to texture ...
g.flush();


NullTexture.java:
Code:
import org.newdawn.slick.opengl.InternalTextureLoader;
import org.newdawn.slick.opengl.Texture;

/**
* A "null" texture which holds no image data and does not
* create or bind any OpenGL textures. This is useful
* for creating off-screen images where there is no need
* to keep hold of the original image (i.e. if the off-screen
* image you are creating is going to be initially empty).
*
* @author davedes
*/
public class NullTexture implements Texture {
   
   private int width, height;
   private int texWidth, texHeight;
   
   public NullTexture(int width, int height) {
      this.width = width;
      this.height = height;
      this.texWidth = InternalTextureLoader.get2Fold(width);
      this.texHeight = InternalTextureLoader.get2Fold(height);
   }

   public boolean hasAlpha() {
      return true;
   }

   public String getTextureRef() {
      return null;
   }

   public void bind() {
   }

   public int getImageHeight() {
      return width;
   }

   public int getImageWidth() {
      return height;
   }

   public float getHeight() {
      return width/(float)texWidth;
   }

   public float getWidth() {
      return height/(float)texHeight;
   }

   public int getTextureHeight() {
      return texWidth;
   }

   public int getTextureWidth() {
      return texHeight;
   }

   public void release() {
   }

   public int getTextureID() {
      return 0;
   }

   public byte[] getTextureData() {
      return null;
   }

   public void setTextureFilter(int textureFilter) {
   }
}




In the long run, I think we should be revisiting the purpose/naming of EmptyImageData (and likewise the Image(width, height) constructor). These are most often used for offscreen rendering, but they result in excessive texture creation. Instead, at least in my opinion, an empty texture should truly be 'empty' in that it stores no RGBA/texture data. If the user really wanted to create a blank texture (without FBO/pbuffer), only then would EmptyImageData be employed.


Top
 Profile  
 
PostPosted: Thu Jun 07, 2012 4:58 pm 
Offline
Game Developer
User avatar

Joined: Thu Mar 03, 2011 6:22 pm
Posts: 534
Hey there davedes. Have you done something on this topic? I think we should really go on making this better. Especially for FBOGraphics.

_________________
Current Projects:
Image Mr. Hat I
Image Vegan Vs. Zombies
Projects:
RadicalFish Engine - Build on top of Slick2D, Ideas, Bugs? Open an Issue ticket!


Top
 Profile  
 
PostPosted: Thu Jun 21, 2012 11:24 am 
Offline
Slick Zombie

Joined: Sat Jan 27, 2007 7:10 pm
Posts: 1469
There is the following static method in Image:
Code:
Image.createOffscreenGraphics


I haven't advertised it much because I'd like to do a "proper fix" at some point but.

Unfortunately now that I've started my summer job I don't have the time. The fix would involve a lot of refactoring -- and it would change the way getGraphics works (acting on the backing texture rather than a copy)


Top
 Profile  
 
PostPosted: Tue Jun 26, 2012 8:04 am 
Offline
Regular

Joined: Sun Oct 30, 2011 4:47 pm
Posts: 184
Location: Mittweida, Saxony, Germany
I looked into the Image/Graphics/FBO-stuff and came to the conclusion that this is nasty. As far as I can see it, there are solutions but its possible I overlooked cases.

Some examples:
1. You got one texture and one image pointing to it. Now you create a second image using
Code:
new Image(Texture)
so you got two images pointing to the same texture. Now you create a FBOGraphics and start drawing onto the second image. FBO creates a new texture and copies the old texture onto it. That is exactly what I would expect. So the first image keeps pointing to the original texture and the second one points to the new texture. In this case also no overhead is created as far as I can see. So for this case the FBO graphics do exactly what I would expect them to do.

2. You want a texture only for the purpose to draw on it and so you create a new image with
Code:
new Image(int, int)
so you get a new image and a new texture the image points to. Now you create your FBOGraphics to draw on that image. In this case the FBO creates a new texture again and overwrites the one stored in the image without cleaning it. In this case the overhead is created, but drawing will work just fine.

In case you change that the FBO cleans up the texture stored in the image it points to, you will get problems in the case described in example one. Because cleaning the texture will render it invalid for the first and the second image. The second image will still work as expected, how ever the first one will be invalid and in case I render this one, expecting the old image to appear, OpenGL will go down the hard way.

The first solution that came to mind are to make the images aware if they "own" the texture. Means if the texture instance was delivered from outside or created inside the image. In case the texture is created in the image, the texture has to be cleaned up once the FBO kicks in, in any other case it has to behave as it does now. That works fine at the first look at it, but in case you switch the roles of the first and the second image in example one, you still got the same problem.

The second solution, and from my current point of view the best one, is to create a internal counter for the textures. So the texture can count how many images refer to the textures. In case this counter drops to zero, the pixeldata of the image can be released.

Did I miss anything?

Nitram

_________________
http://illarion.org


Top
 Profile  
 
PostPosted: Tue Jun 26, 2012 10:46 am 
Offline
Game Developer
User avatar

Joined: Thu Mar 03, 2011 6:22 pm
Posts: 534
No what you want to do would work with normal Slick. But it would lead to problems with Slick-Util. Because some don't eben the Image class. So if you injected code in the texture to release the data (or in the Loader) you gonna have a bad time :D

Imho if I create an Image with a texture reference, it should be clear that it always uses this texture as reference.

Also I nice idea would be to do it like TWL does. There ouy push new data to the texture. This would be a good fallback in case both PBuffer and FBO's fail.

_________________
Current Projects:
Image Mr. Hat I
Image Vegan Vs. Zombies
Projects:
RadicalFish Engine - Build on top of Slick2D, Ideas, Bugs? Open an Issue ticket!


Top
 Profile  
 
PostPosted: Tue Jun 26, 2012 2:25 pm 
Offline
Slick Zombie

Joined: Sat Jan 27, 2007 7:10 pm
Posts: 1469
Nitram wrote:
I looked into the Image/Graphics/FBO-stuff and came to the conclusion that this is nasty. As far as I can see it, there are solutions but its possible I overlooked cases.

Yup, it's pretty nasty. :|

Quote:
1. You got one texture and one image pointing to it. Now you create a second image using
Code:
new Image(Texture)
so you got two images pointing to the same texture. Now you create a FBOGraphics and start drawing onto the second image. FBO creates a new texture and copies the old texture onto it. That is exactly what I would expect. So the first image keeps pointing to the original texture and the second one points to the new texture. In this case also no overhead is created as far as I can see. So for this case the FBO graphics do exactly what I would expect them to do.

I don't think copying should always be the expected functionality. Generally we are working on shallow copies of images that share the same backing texture; so it would make more sense to keep that "shallowness" consistent when dealing with getGraphics as well. Code example:
Code:
//typical sprite sheet usage:
Image sheet = new Image(tex);
Image sub1 = sheet.getSubImage(x, y, w, h);
Image sub2 = sheet.getSubImage(x2, y2, w2, h2);

//at this point we can operate on "tex" and changes will be reflected in both sub images
tex.setTextureFilter(...);

//now let's say we want to modify a sprite in our sheet, so we use getGraphics.
sub1.getGraphics() ... flush() etc

//now sub1 has its own Texture object, so modifying "tex" no longer affects it.. e.g.
tex.setTextureFilter(...); <-- only changes sub1

//"sheet " can also no longer be used as a sprite sheet with sub1/sub2 since the texture is different
sheet.startUse();
sub1.drawEmbedded(...); <-- causes error
sheet.endUse();

//also we can't use things like glTexImage2D on "tex" and expect it to be reflected in all sub images


IMO it would be better to force the user to make a copy -- that way they know exactly when they are dealing with a copy, and when they are dealing with a shared texture. Here is what I propose:
Code:
Image sheet = new Image(sheetTex);

//this is a SHALLOW copy!!! does not copy texture
Image sub1 = sheet.getSubImage(x, y, w, h);
Image sub2 = sheet.copy(); //this is also a shallow copy

//getGraphics should modify the SHARED texture rather than a copy (as we would expect with a "shallow" copy)

//if we want to modify a copy of the texture then we need to do something like this:
Image sub1Copy = new Image(sub1.getWidth(), sub1.getHeight());
Graphics g = sub1Copy.getGraphics();
g.drawImage(sub1); //draw sub1 to sub1Copy
g.flush();

//now we know that sub1Copy does NOT share the same backing texture as sub1/sub2
//whereas we can still use sub1/sub2 in line with sheet.startUse/endUse


Quote:
The second solution, and from my current point of view the best one, is to create a internal counter for the textures. So the texture can count how many images refer to the textures. In case this counter drops to zero, the pixeldata of the image can be released.

This will solve the memory leak, but in the end we are still getting more textures created than necessary. The root of the problem is that we are creating new textures behind the scenes, instead of sharing them between shallow Image copies.

And very often a user will want to act on a Texture without using an Image -- for e.g. in shaders, or 3D, or Slick-Util, etc. Coupling Image and Texture would severely limit the capabilities of Slick as a "generic" OpenGL utility library.


Top
 Profile  
 
PostPosted: Wed Jun 27, 2012 9:19 am 
Offline
Regular

Joined: Sun Oct 30, 2011 4:47 pm
Posts: 184
Location: Mittweida, Saxony, Germany
If we work with shallow and real copies here, we really need to create functions that identify exactly the kind of copy created. I would actually expect image.copy to create a real copy to a image/texture.

I also worry about breaking existing games here. Because if you use RTT currently you always create a real copy once the graphics object is created. So all other images referring to the same texture are uneffected (as I actually expected it) by the drawing operations.

We need to define exactly what is to happen when before starting to fix anything. Else its getting inconsistent again.

EDIT:
Another question: Is the aim of the first release still to keep binary-level backward compatiblity to older versions of Slick?

Nitram

_________________
http://illarion.org


Top
 Profile  
 
PostPosted: Wed Jun 27, 2012 11:37 pm 
Offline
Slick Zombie

Joined: Sat Jan 27, 2007 7:10 pm
Posts: 1469
You only expect them to return hard copies (with copied pixel data) because of the name "Image" and Java2D convention. "Image" is a bit misleading as it implies data storage; in fact it should have been called TextureRegion. It doesn't hold any information about the image data; instead it just holds texture coordinates, size, and an associated Texture object. Image has always worked this way, although maybe it was never clarified in documentation.

Shallow copies are fast and generally preferred when you need to flip or scale an image, or when you need a sub image. The only time you should copy Texture data is when you need to modify the Texture but retain the original as well. For e.g. say you are making multiple copies of a sprite sheet, each with different colours. This requires multiple Texture objects. On the other hand, if you only have one sprite sheet and you want to alter the colours of certain sub-images, you will not want to copy the texture data.

IMO it was a design error to make getGraphics return a copy; and that error has lead to the memory leak, unnecessary texture creation, and other issues. Image has always been a thin wrapper around Texture and nothing more; making getGraphics copy the texture data is inconsistent with the rest of Image's "shallowness." If it's not clear that Image is shallow, we need to improve the docs.

Breaking old games is inevitable when you fix a bug that's been around for so long. Ideally, though, our new API will be as similar to our old one in terms of code -- which is why I feel a "proper" fix is in order. So "new Image(256, 256)" followed by "getGraphics()" will only result in a single texture being created. We shouldn't worry too much about binary compatibility (although obviously minimizing breakages is preferred) as long as the code style remains consistent.

Just my 2c.


Top
 Profile  
 
PostPosted: Thu Jun 28, 2012 8:16 am 
Offline
Regular

Joined: Sun Oct 30, 2011 4:47 pm
Posts: 184
Location: Mittweida, Saxony, Germany
Okay.

In this case we should try to make it "clear" that the Image is only a wrapper. In my opinion that is done best by throwing away the constructor Image(int, int) and all similar constructors as this one creates a texture and gives the Image this "heavy" appearance.

To create new textures I'd use the TextureLoader class with additional functions like ... I don't know... createTexture ? :wink:
To give a clear impression that at this point a new texture is created.

Furthermore I would not allow the texture a Image points to to be changed. To avoid future problems. As a image is only a light-weight wrapper you can easily create new instances as needed.
This changes would also greatly reduce the amount of weird code inside the image class and is in my opinion that most clean way to do it.

If acceptable I would look into trying to implement this.

Nitram

_________________
http://illarion.org


Top
 Profile  
 
PostPosted: Thu Jun 28, 2012 11:18 am 
Offline
Slick Zombie

Joined: Sat Jan 27, 2007 7:10 pm
Posts: 1469
I don't think we should throw away those constructors for two reasons: (a) users are used to them, (b) they exist for convenience. Slick is meant to be a "convenient" and easy to use library; there are places where we sacrifice "perfect OO design" for the sake of simplicity.

All of these constructors exist for convenience, so that your code looks like this...
Code:
//this creates a new texture
img = new Image(25, 25);

//this also creates a new texture
img2 = new Image("res/blah.png");

//yet another texture-creating constructor
img3 = new Image(myImageData);


Instead of this...
Code:
try {
    Texture tex = InternalTextureLoader.get().createTexture(25, 25, Image.FILTER_NEAREST);
    img = new Image(tex);
} catch (IOException e) {
    throw new SlickException("error loading image", e);
}


I think a combination of forum posts and documentation (JavaDocs and Wiki) would help clarify the change we'll be making to Image.


Top
 Profile  
 
PostPosted: Thu Jun 28, 2012 11:57 am 
Offline
Regular

Joined: Sun Oct 30, 2011 4:47 pm
Posts: 184
Location: Mittweida, Saxony, Germany
The problem I see here is that the "simplicity" leads in this case to confusion.

And I'm talking more about something like:

Code:
Image img = new Image(TextureLoader.createImage(25, 25));


And this is not that bad in my opinion.

Also... throwing it away is maybe a little harsh. But I think about flagging it as "deprecated" at least to mark that this is a at some point "dirty" solution.

Nitram

_________________
http://illarion.org


Top
 Profile  
 
PostPosted: Thu Jun 28, 2012 1:11 pm 
Offline
Game Developer
User avatar

Joined: Thu Mar 03, 2011 6:22 pm
Posts: 534
It's maybe not bad for you. But Slick2D was always a starting point for starters with OpenGL and Java (using LWJGL of course). We need to be sure that the newbies are not filled with strange Constructors. We want the user to have it simple. And what you do is also not good programming style. You enforce a useless call to the user. Image can create the texture too.

Your are right, we have a lot of methods. But that's a Framework not a direct game so we probalty want to make it as flexible as we can make it. The Java API works the same way :)

_________________
Current Projects:
Image Mr. Hat I
Image Vegan Vs. Zombies
Projects:
RadicalFish Engine - Build on top of Slick2D, Ideas, Bugs? Open an Issue ticket!


Top
 Profile  
 
PostPosted: Thu Jun 28, 2012 1:28 pm 
Offline
Regular

Joined: Sun Oct 30, 2011 4:47 pm
Posts: 184
Location: Mittweida, Saxony, Germany
Actually the Java API does not work this way. In the Java API the classes have clear tasks and close to no overlapping.

What is it the Slick image does?
  • it wraps textures (its main task)
  • it draws (parts of) textures
  • it creates textures
  • anything else?

Thats not "simple", its confusing.
What is the difference between "Graphics.drawImage()" and "Image.draw()"?
If we create multiple methods to do the same thing and create classes that do everything but cooking coffee, its just plain confusing because no one understands the side effects and the differences between the different functions.

"Simple" for me is defined by clear ways how to do things. And not be offering multiple ways to do the same thing.
You want a shallow copy of a texture or a part of it? Use the Image class!
You want a real copy of a texture or a part of it? Use the Texture class!
You want to draw something? Use the Graphics class!

Now its like...
You want to draw something? Use the Graphics class! Or the Image class! But its faster to use the drawEmbedded stuff! Ah just take the texture and write the OpenGL stuff yourself!

You see my point? I don't see simplicity in this approach... I see confusion and multiple implementations of the very same thing.

Nitram

_________________
http://illarion.org


Top
 Profile  
 
PostPosted: Thu Jun 28, 2012 3:43 pm 
Offline
Slick Zombie

Joined: Sat Jan 27, 2007 7:10 pm
Posts: 1469
Slick is "broken by design" in many ways. I would say that your Texture.createTexture() suggestion is better in terms of design, but it has a couple issues:

1. It forces users to write things in a different way. 99% of Slick users will copy-paste from tutorials and old forum posts; ideally we should let them do that by not breaking or deprecating old code when possible.
2. It introduces more duplication in InternalTextureLoader. We already have a createTexture (instance) method that throws IOException; now you are suggesting a static method that throws SlickException.
3. InternalTextureLoader and TextureImpl will probably be deprecated eventually; so we shouldn't be designing around it. They were never really intended for public use (hence Kev's creation of TextureLoader to wrap InternalTextureLoader, but that too could go). Ideally we should do something like this:
Code:
//A: create an empty texture
Texture tex = new Texture(256, 256, Texture.FILTER_LINEAR);
//B: pass byte buffer to create a texture
Texture tex2 = new Texture(imgData);
//C: convenience method to load texture from file
Texture tex3 = Texture.loadTexture("res/img.png", Texture.FILTER_LINEAR);

This would stay within the bounds of Slick's "baby style" and get rid of the clutter/issues with TextureImpl/InternalTextureLoader. It wouldn't change things dramatically as most people don't rely on InternalTextureLoader anyways. And of course it would be easy to keep those old classes around for backward compatibility. However, this is a big change that would be better suited for future releases.

Slick is not perfect nor is it beautifully designed. But it is a good playground for a beginner to get a small game going, which is exactly the audience it targets. This is why this bug has gone unnoticed for so many years -- none of the users noticed or cared that an extra texture was getting created! In fact, most users will probably never need to think about Texture -- as they only need to deal with Images.

Regarding drawEmbedded, it would be nice to have Image.draw automatically forward calls to startUse/drawEmbedded/endUse, but that's not possible without breaking all of Slick. Hooray for immediate mode GL...

Slick is many years old and was made back when immediate mode was more common. "Properly fixing" Slick means re-writing it completely from the ground up, tailoring it to "modern OpenGL" (shaders and the programmable pipeline), and turning it into a different library altogether. I've been doing something similar in my spare time, although progress is slow:
https://github.com/mattdesl/slim/tree/m ... im/texture


Top
 Profile  
 
PostPosted: Thu Jun 28, 2012 4:35 pm 
Offline
Regular

Joined: Sun Oct 30, 2011 4:47 pm
Posts: 184
Location: Mittweida, Saxony, Germany
If Texture or TextureLoader creates the texture is not really important from my point of view. Maybe the Texture class is even better. I was avoiding to propose the usage of the InternalTextureLoader because... as the name says, its internal. Reads for me: Not for use inside a application.

While I understand that you don't want to break the ancient examples, it just pains me to see that there are no plans to drag Slick out of its current rather broken state... :(

Nitram

_________________
http://illarion.org


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 16 posts ]  Go to page 1, 2  Next

All times are UTC


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group