Slick Forums

Discuss the Slick 2D Library
It is currently Fri May 24, 2013 4:09 pm

All times are UTC




Post new topic Reply to topic  [ 7 posts ] 
Author Message
PostPosted: Wed Mar 10, 2010 10:19 am 
Offline

Joined: Tue Jan 12, 2010 11:49 am
Posts: 10
I have a source for video-image data which shall be rendered to a Slick-screen. Now I am thinking about how to render these images as efficiently as possible. My first naive attempt to just set each pixel by its RGB value was quite slow. Does anyone know a better way?

As the video source offers a way to get an AWT BufferedImage there would also the option to directly render the Image to slick's graphical context. Is this possibility given?


Top
 Profile  
 
 Post subject:
PostPosted: Wed Mar 10, 2010 10:53 pm 
Offline
Regular

Joined: Tue Mar 03, 2009 11:53 pm
Posts: 123
If you can get your data in a ByteBuffer, you can try this OpenGL code

Code:
Texture t = immagine.getTexture();
            t.bind();
            GL11.glTexParameteri(GL11.GL_TEXTURE_2D, GL11.GL_TEXTURE_MIN_FILTER, SGL.GL_LINEAR);
            GL11.glTexParameteri(GL11.GL_TEXTURE_2D, GL11.GL_TEXTURE_MAG_FILTER, SGL.GL_LINEAR);

            // produce a texture from the byte buffer
            GL11.glTexImage2D(GL11.GL_TEXTURE_2D,
                                0,
                                GL11.GL_RGBA,
                                this.width,
                                this.height,
                                0,
                                GL11.GL_RGBA,
                                GL11.GL_UNSIGNED_BYTE,
                                this.ii);


this.ii is a ByteBuffer, which I copy directly into a texture. It's fast, I've been using it for rendering movies real-time.

ciao!

_________________
Carotinho


Top
 Profile  
 
 Post subject:
PostPosted: Thu Mar 11, 2010 11:49 am 
Offline

Joined: Tue Jan 12, 2010 11:49 am
Posts: 10
Thx! You mentioned to use this for rendering movies. As I had several problem to find a playback framework for Java (currently it is xuggler, which is quite a bit over the top) - what are you using?


Top
 Profile  
 
 Post subject:
PostPosted: Thu Mar 11, 2010 8:36 pm 
Offline
Regular

Joined: Tue Mar 03, 2009 11:53 pm
Posts: 123
I've used gstreamer, through gstreamer-java. Works well and quite easy to setup. I can fish out some code, if you want.

ciao!

_________________
Carotinho


Top
 Profile  
 
 Post subject:
PostPosted: Fri Mar 12, 2010 12:46 am 
Offline

Joined: Tue Jan 12, 2010 11:49 am
Posts: 10
Now I had the time to test the code - but my results are not too satisfiying.

my test code looks now like this
Code:
Texture texture = null;
try
{
  image = new Image(1024, 768);
}
catch(Exception e)
{
  e.printStackTrace();
}
               
texture = image.getTexture();
texture.bind();
GL11.glTexParameteri(GL11.GL_TEXTURE_2D,     GL11.GL_TEXTURE_MAG_FILTER, SGL.GL_LINEAR);

// produce a texture from the byte buffer
GL11.glTexImage2D(GL11.GL_TEXTURE_2D,
                                         0,
                                         GL11.GL_RGBA,
                                         1024, 768,
                                         0,
                                         GL11.GL_RGBA,
                                         GL11.GL_UNSIGNED_BYTE,
                                         newPic.getByteBuffer());
                  
g.drawImage(image, 0, 0);


The video has a resolution of 1024x768, but my example throws the exception:

Code:
java.lang.IllegalArgumentException: Number of remaining buffer elements is 2359296, must be at least 3145728
   at org.lwjgl.BufferChecks.throwBufferSizeException(BufferChecks.java:125)
   at org.lwjgl.BufferChecks.checkBufferSize(BufferChecks.java:140)
   at org.lwjgl.NondirectBufferWrapper.wrapBuffer(NondirectBufferWrapper.java:101)
   at org.lwjgl.opengl.GL11.glTexImage2D(GL11.java:2721)
   at test.XuggleVideo.update(XuggleVideo.java:258)
   at test.VideoTest.render(VideoTest.java:59)
   at org.newdawn.slick.GameContainer.updateAndRender(GameContainer.java:681)
   at org.newdawn.slick.AppGameContainer.gameLoop(AppGameContainer.java:408)
   at org.newdawn.slick.AppGameContainer.start(AppGameContainer.java:318)
at test.VideoTest.main(VideoTest.java:96)


When I switch the resolution e.g. to 800x600, the application starts, but the picture is massively disturbed (because of the wrong resolution?).
As I have nearly no knowledge about configuring the texture context of openGL, I do not know if your code is applicable to my video.

Can you give me some hints?


Top
 Profile  
 
 Post subject:
PostPosted: Sun Mar 14, 2010 5:10 pm 
Offline
Regular

Joined: Tue Mar 03, 2009 11:53 pm
Posts: 123
You are creating a 1024x768x3 buffer, somewhere: it should be 1024x768x4 to match the size.

I'm going to search for my code and post it later.

Ciao!

_________________
Carotinho


Top
 Profile  
 
 Post subject:
PostPosted: Sun Mar 14, 2010 5:35 pm 
Offline
Regular

Joined: Tue Mar 03, 2009 11:53 pm
Posts: 123
Ok here's the file.

Code:
import java.io.File;
import java.nio.ByteBuffer;
import java.nio.IntBuffer;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import org.gstreamer.Bus;
import org.gstreamer.Caps;
import org.gstreamer.ClockTime;
import org.gstreamer.Gst;
import org.gstreamer.GstObject;
import org.gstreamer.StreamInfo;
import org.gstreamer.elements.PlayBin;
import org.gstreamer.elements.RGBDataAppSink;
import org.lwjgl.opengl.GL11;
import org.newdawn.slick.Image;
import org.newdawn.slick.opengl.ImageData;
import org.newdawn.slick.opengl.Texture;
import org.newdawn.slick.opengl.renderer.SGL;

/**
*
* @author dario
*/
public class GSVideo1 implements ImageData, RGBDataAppSink.Listener  {



    private class RGBListener implements RGBDataAppSink.Listener {

        //private int contatore;

        public void rgbFrame(int movieWidth, int movieHeight, IntBuffer arg2) {
            // Qui devo farci qualcosa con i frames...

            //System.out.println("width " + movieWidth);
            //System.out.println("height " + movieHeight);

            //contatore ++;
            //System.out.println("Frame " + contatore);
            newFrame(movieWidth, movieHeight, arg2);

           
        }
    }

    private class EOSListener implements Bus.EOS {

        public void endOfStream(GstObject arg0) {
            EOSEvent();
        }
    }

    private class myThread implements Runnable {

        public void run() {
            myPlayer.play();
        }

    }


    private Image immagine;
    private int width;
    private int height;
    private int widthStep;
    private int target;
    private int filter;
    private String nomeFile;
    //private PlayBin2 myPlayer;
    private PlayBin myPlayer;
    private ByteBuffer bb;
    private IntBuffer ii;
    private int[] buffer;
    private Boolean frameReady;
   
    private RGBDataAppSink videoSink;
    private RGBDataAppSink.Listener myListener;
    private Bus.EOS myEOSListener;

    private Boolean finito;




    static Boolean isInit = false;
    static int refCount = 0;

    static void init() {
        if (!isInit) {
            //Gst.init();
            String[] args = new String[1];
            args[0] = "";
            Gst.init("Ciao", args);

            System.out.println("Gst.init");
        }

        refCount++;
    }

    static void deInit() {
        refCount--;
        if (refCount == 0) {
            System.out.println("Gst.deinit");
            Gst.deinit();
        }
    }

    public GSVideo1(String nomeFile, int width, int height) {

        init();

        this.nomeFile = nomeFile;
        File f = new File(nomeFile);

        //myPlayer = new PlayBin2("DarioMoviePlayer");
        myPlayer = new PlayBin("DarioMoviePlayer");
        myPlayer.setInputFile(f);

       


       
        myListener = new RGBListener();
        myEOSListener = new EOSListener();

        // Provo con il sink
        //videoSink = new RGBDataAppSink("rgb", myListener);
        videoSink = new RGBDataAppSink("rgb", this);
        videoSink.setPassDirectBuffer(true);

        Caps c = videoSink.getCaps();
        System.out.println("Caps: " + c);

        // Imposto questo sink alla pipeline
        myPlayer.setVideoSink(videoSink);

        // Connetto il bus
        myPlayer.getBus().connect(myEOSListener);

        myPlayer.pause();

        /*
        // Lista delle proprietà dello stream
        List lista = myPlayer.getStreamInfo();
       
        Iterator<StreamInfo> iter = lista.iterator();
        while (iter.hasNext()) {
            StreamInfo s = iter.next();
            System.out.println(s.toString());           
        }
         */


        this.width = width;
        this.height = height;

        this.target = GL11.GL_TEXTURE_2D;
        this.filter = GL11.GL_NEAREST;

        buffer = new int[width * height];
        ii = IntBuffer.allocate(width * height);

        immagine = new Image(this);

        frameReady = false;
        finito = false;
    }

    public void play() {
        //new Thread(new myThread()).start();
        myPlayer.play();
    }

    public void pause() {
        myPlayer.pause();
    }

    public void stop() {
        myPlayer.stop();
    }

    public void rewind() {
        myPlayer.seek(ClockTime.ZERO);
    }

    public float duration() {
        float sec = myPlayer.queryDuration().toSeconds();
        float nanosec = myPlayer.queryDuration().getNanoSeconds();
        return sec + nanoSecToSecFrac(nanosec);
    }


    public synchronized void update(int delta) {
        if (this.frameReady) {
            this.frameReady = false;

            ii.rewind();
           
            // Copio i dati nella texture, in qualche modo
            Texture t = immagine.getTexture();
            t.bind();
            GL11.glTexParameteri(target, GL11.GL_TEXTURE_MIN_FILTER, SGL.GL_LINEAR);
            GL11.glTexParameteri(target, GL11.GL_TEXTURE_MAG_FILTER, SGL.GL_LINEAR);

            // produce a texture from the byte buffer
            GL11.glTexImage2D(target,
                                0,
                                GL11.GL_RGBA,
                                this.width,
                                this.height,
                                0,
                                GL11.GL_RGBA,
                                GL11.GL_UNSIGNED_BYTE,
                                this.ii);
             
        }

    }

    public Image getImage() {
        return this.immagine;
    }

    public void draw(float x, float y) {
        this.immagine.draw(x, y);
    }

    public void distruggi() {
        System.out.println("Distruzione di GSVideo");
        deInit();
    }

    public int getDepth() {
        return 32;
    }

    public int getWidth() {
        return this.width;
    }

    public int getHeight() {
        return this.height;
    }

    public int getTexWidth() {
        return this.width;
    }

    public int getTexHeight() {
        return this.height;
    }

    public ByteBuffer getImageBufferData() {
        return this.bb;
    }


    protected void sync() {
       

    }

    private synchronized void newFrame(int movieWidth, int movieHeight, IntBuffer arg2) {
        rgbFrame(movieWidth, movieHeight, arg2);
    }

    public synchronized void rgbFrame(int arg0, int arg1, IntBuffer arg2) {
        //ii = arg2;

        ii.rewind();
        ii.put(arg2);
        frameReady = true;
    }

    private void EOSEvent() {
        finito = true;
    }

    private Boolean isFinito() {
        return finito;
    }

    private float nanoSecToSecFrac(float nanosec)
    {
        for (int i = 0; i < 3; i++) nanosec /= 1E3;
        return nanosec;
    }
}


What I do is basically starting a gstreamer pipeline through it's convenient PlayBin. I create a new RGBSink, which can act as a "receiver" of data. When data arrives, I copy them somewhere safe and when updating I texturize them. A better way could be to directly texturize when a new frame arrives.
This class implements ImageData, so that I can create an image according to my needs (32 bit depth, that is 4 bytes).
The size of the movie is passed to the constructor only because at the time I couldn't find a way to automatically detect it when creating the playbin. But I'm sure I missed something:)

I have to note that incoming data had R and B channels swapped, so I wrote a little shader to invert them when rendering (easier than actually discovering WHY:D)

Ciao!

_________________
Carotinho


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 7 posts ] 

All times are UTC


Who is online

Users browsing this forum: No registered users and 0 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group