首页 > 代码库 > Android4.0 Surface机制分析

Android4.0 Surface机制分析

1. java层面的Surface

    对于Surface我们的认识主要是android的类Surface, android的文档描述Surface是“Handle onto a raw buffer that is being managed by the screen compositor”,这个描述透漏出两个信息:首先,Surface是一个raw buffer的句柄,通过它去管理一个raw buffer,其次,Surface本身是由screen compositor来管理的。但是raw buffer具体是什么,screen compositor又是什么,Surface是如何管理一个raw buffer,而它又是怎样被compositor来管理,后续我们会具体来分析。

    Surface的具体使用上,我们通常并不直接去手动创建一个Surface,尽管可以这么做,通常,我们是通过SurfaceView,GLSurfaceView间接地去创建Surface,当然,更多的时候我们在使用Surface而不自知,在Andorid的窗口实现里,每一个Window其实都会对应一个Surface,而每个Activity都会持有一个Window,所以,我们通常在Activity里设置的view(通过setContentView),从java抽象上看其最终的绘制目标就是在Surface上。(Window以及view本身的结构也比较复杂,我会在后面的文章里再讨论这个部分。)

    下面我们来看看Surface的接口都有什么,除了重要的构造函数外,比较重要的两个接口,lockCanvas, unlockCanvasAndPost. lockCanvas会返回一个Canvas给我们,我们可以使用这个Canvas来改变Surface的内容, 但是这个改变不会立刻生效,只有在我们调用了unlockCanvasAndPost之后,改变才会生效。

    上面说了,通常我们并不手动去创建Surface,帮我们创建Surface的是WindowManagerService,它在帮我们创建Surface时,会生成一个SurfaceSession,然后将这个SurfaceSession作为参数来构造Surface。SurfaceSession,其代表的是与前面说的compositor的一个会话连接。

image

      

2. 一些重要的类及关系介绍

   在开始介绍cpp层面的Surface工作流程之前,我们先简要的介绍几个比较重要的类,因为Surface的机制抽象层级比较多,对基本类的功能有一个初步了解有助于我们更容易理解。

   首先,需要说明的是,java层的Surface与cpp层的Surface并不相同,他们之间存在着关系,但并不是同一个抽象. 而cpp层里Surface与ISurface又是不同的抽象,切忌混为一谈。

   我们知道,在android里大量的使用了CS模式,把一些功能做成服务Service,供各个客户端Client访问,而Client,Service都具有相同的接口,这样抽象掉了Client,Service之间的通信,让Client,Service各自独立,消除耦合。对于Surface部分来说,Android的framework里主要涉及到以下的几个接口:

SurfaceFinger:这个是Surface服务端的总管家,它具体是ISurfaceComposer接口的服务端实现。

ComposerService:这个是为客户端取得ISurfaceComposer代理而提供的方便类。

ISurfaceComposer:通过这个接口可以访问到SurfaceFlinger,可以通过它建立一个会话,即ISurfaceComposerClient,也可以通过它去更新Surface的相关信息,这个是通过setTransactionState接口完成的。

SurfaceFlinger::Client: ISurfaceComposerClient的服务端实现。

SurfaceComposerClient: 持有ISurfaceComposerClient的客户端代理,在SurfaceComposerClient初次实例化时,通过ISurfaceComposer的createConnection()接口得到一个ISurfaceComposerClient的代理。同时,它也会管理Surface的状态,通过ISurfaceComposer更新Surface状态,状态的具体保存涉及到一个相关类Composer。可以说,它是Surface跟服务端打交道一个非常重要的接口。

ISurfaceComposerClient:代表一个到SurfaceFinger的会话连接。

SurfaceControl:从字面上看,其作用是控制Surface。其实际作用是持有ISurface的代理及SurfaceComposerClient

ISurfaceTexture:其对应具体的buffer管理

SurfaceTextureClient: 持有ISurfaceTexture的本地代理,通过它可以访问到ISurfaceTexture的实现。同时,特别注意的是,它继承了ANativeWindow,而Surface类会继承SurfaceTextureClient. ANativeWindow代表的是本地窗口,在创建EGL的eglSurface时需要用到它。

 

3. cpp层面的Surface

    从前面的图中可见,在Surface及SurfaceSession的构造过程中,都会调用到各自的init方法,而init的具体实现是在cpp层,具体对framework/base/core/jni/android_view_Surface.java.

我们来看看这两个init的代码:

static void SurfaceSession_init(JNIEnv* env, jobject clazz)
{
    sp<SurfaceComposerClient> client = new SurfaceComposerClient;
    client->incStrong(clazz);
    env->SetIntField(clazz, sso.client, (int)client.get());
}
<style type="text/css"> .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; }</style>

SurfaceSession的init方法SurfaceSession_init主要是获取到一个SurfaceComposerClient,这个类是非常重要的与服务端通讯的类,它的作用:

  • 通过在构造函数里使用ISurfaceComposer的createConnection接口,它获得ISurfaceComposerClient的本地代理
  • 通过得到的ISurfaceComposerClient创建或销毁ISurface
  • 通过它持有的Composer来记录Surface的状态,并在Composer里通过ISurfaceComposer接口更新状态。
    
static void Surface_init(
        JNIEnv* env, jobject clazz,
        jobject session,
        jint, jstring jname, jint dpy, jint w, jint h, jint format, jint flags)
{
    if (session == NULL) {
        doThrowNPE(env);
        return;
    }
 
    SurfaceComposerClient* client =
            (SurfaceComposerClient*)env->GetIntField(session, sso.client);
 
    sp<SurfaceControl> surface;
    if (jname == NULL) {
        surface = client->createSurface(dpy, w, h, format, flags);
    } else {
        const jchar* str = env->GetStringCritical(jname, 0);
        const String8 name(str, env->GetStringLength(jname));
        env->ReleaseStringCritical(jname, str);
        surface = client->createSurface(name, dpy, w, h, format, flags);
    }
 
    if (surface == 0) {
        jniThrowException(env, OutOfResourcesException, NULL);
        return;
    }
    setSurfaceControl(env, clazz, surface);
}
<style type="text/css"> .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; }</style>

sp<SurfaceControl> SurfaceComposerClient::createSurface(
        const String8& name,
        DisplayID display,
        uint32_t w,
        uint32_t h,
        PixelFormat format,
        uint32_t flags)
{
    sp<SurfaceControl> result;
    if (mStatus == NO_ERROR) {
        ISurfaceComposerClient::surface_data_t data;
        sp<ISurface> surface = mClient->createSurface(&data, name,
                display, w, h, format, flags);
        if (surface != 0) {
            result = new SurfaceControl(this, surface, data);
        }
    }
    return result;
}
<style type="text/css"> .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; }</style>

sp<Surface> SurfaceControl::getSurface() const
{
    Mutex::Autolock _l(mLock);
    if (mSurfaceData =http://www.mamicode.com/= 0) {
        sp<SurfaceControl> surface_control(const_cast<SurfaceControl*>(this));
        mSurfaceData = http://www.mamicode.com/new Surface(surface_control);
    }
    return mSurfaceData;
}
<style type="text/css"> .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; }</style>

而Surface_init主要是通过SurfaceComposerClient获取了一个sp<SurfaceControl>,SurfaceControl是对ISurface及Surface的一层封装,其需要注意的地方是:

  • 通过构造SurfaceControl时传入ISurface,其持有ISurface接口
  • 它的getSurface方法可以构造一个Surface对象,并通过mSurfaceData持有Surface对象
  • 它可以通过SurfaceComposerClient去更改Surface的状态

通过上面的分析我们得到了两个Surface,sp<ISurface>, sp<Surface>,前者,我们知道是ISurface的本地代理,但是Surface是做什么的呢,我们下面将具体分析Surface。

4. Surface对象

前面我们提到SurfaceControl的getSurface方法会构造一个Surface对象,事实上,Surface是我们Android GUI系统的核心部分,我们来看看代码:

Surface::Surface(const sp<SurfaceControl>& surface)
    : SurfaceTextureClient(),
      mSurface(surface->mSurface),
      mIdentity(surface->mIdentity)
{
    sp<ISurfaceTexture> st;
    if (mSurface != NULL) {
        st = mSurface->getSurfaceTexture();
    }
    init(st);
}
void Surface::init(const sp<ISurfaceTexture>& surfaceTexture)
{
    if (mSurface != NULL || surfaceTexture != NULL) {
        LOGE_IF(surfaceTexture==0, "got a NULL ISurfaceTexture from ISurface");
        if (surfaceTexture != NULL) {
            setISurfaceTexture(surfaceTexture);
            setUsage(GraphicBuffer::USAGE_HW_RENDER);
        }
 
        DisplayInfo dinfo;
        SurfaceComposerClient::getDisplayInfo(0, &dinfo);
        const_cast<float&>(ANativeWindow::xdpi) = dinfo.xdpi;
        const_cast<float&>(ANativeWindow::ydpi) = dinfo.ydpi;
        const_cast<uint32_t&>(ANativeWindow::flags) = 0;
    }
}
<style type="text/css"> .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; }</style>

可以看到,在构造函数里,通过ISurface接口的getSurfaceTexture,我们得到一个ISurfaceTexture的本地代理,在init里我们保存了该代理,即SurfaceTextureClient将持有ISurfaceTextureClient。

我们第一节里说java层面的Surface有两个接口,lockCanvas,unlockCanvasAndPost,我们来看看其底层实现是怎样的

   const sp<Surface>& surface(getSurface(env, clazz));
      ........................................
   status_t err = surface->lock(&info, &dirtyRegion);
 
static void Surface_unlockCanvasAndPost(
        JNIEnv* env, jobject clazz, jobject argCanvas)
{
    jobject canvas = env->GetObjectField(clazz, so.canvas);
    if (env->IsSameObject(canvas, argCanvas) == JNI_FALSE) {
        doThrowIAE(env);
        return;
    }
 
    const sp<Surface>& surface(getSurface(env, clazz));
    if (!Surface::isValid(surface))
        return;
 
    // detach the canvas from the surface
    SkCanvas* nativeCanvas = (SkCanvas*)env->GetIntField(canvas, no.native_canvas);
    int saveCount = env->GetIntField(clazz, so.saveCount);
    nativeCanvas->restoreToCount(saveCount);
    nativeCanvas->setBitmapDevice(SkBitmap());
    env->SetIntField(clazz, so.saveCount, 0);
 
    // unlock surface
    status_t err = surface->unlockAndPost();
    if (err < 0) {
        doThrowIAE(env);
    }
}
<style type="text/css"> .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; }</style>
可以看到,lockCanvas最终会调用到Surface的lock方法,而unlockCanvasAndPost最终调用到Surface的Surface的unlockAndPost,前面说到lockCanvas会返回一个Canvas供我们绘制使用,这个Canvas其实是利用lock得到的缓冲区来构建的。我们下面来分析。

首先,我们说Surface是一个ANativeWindow,它的继承关系如下:

image

ANativeWindow我们前面说过,它其实是一个EGL可以操作的窗口,其具体定义在system/core/include/system/window.h里,它的主要接口有dequeueBuffer,queueBuffer,lockBuffer等。

好了,搞清楚Surface是个什么东西,下面来看其lock及unlockAndPost方法:

status_t Surface::lock(SurfaceInfo* other, Region* inOutDirtyRegion) {
    ANativeWindow_Buffer outBuffer;
 
    ARect temp;
    ARect* inOutDirtyBounds = NULL;
    if (inOutDirtyRegion) {
        temp = inOutDirtyRegion->getBounds();
        inOutDirtyBounds = &temp;
    }
 
    status_t err = SurfaceTextureClient::lock(&outBuffer, inOutDirtyBounds);
 
    if (err == NO_ERROR) {
        other->w = uint32_t(outBuffer.width);
        other->h = uint32_t(outBuffer.height);
        other->s = uint32_t(outBuffer.stride);
        other->usage = GRALLOC_USAGE_SW_READ_OFTEN | GRALLOC_USAGE_SW_WRITE_OFTEN;
        other->format = uint32_t(outBuffer.format);
        other->bits = outBuffer.bits;
    }
 
    if (inOutDirtyRegion) {
        inOutDirtyRegion->set( static_cast<Rect const&>(temp) );
    }
 
    return err;
}
<style type="text/css"> .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; }</style>
status_t Surface::unlockAndPost() {
    return SurfaceTextureClient::unlockAndPost();
}

这两个方法都会调用到SurfaceTextureClient里的相应方法,代码这里就不贴了,lock的实现主要是dequeueBuffer出一个GraphicBuffer,而unlockAndPost主要是将这个GraphicBuffer入队queueBuffer。我们来看看SurfaceTextureClient的dequeueBuffer及queueBuffer实现:

int SurfaceTextureClient::dequeueBuffer(android_native_buffer_t** buffer) {
    LOGV("SurfaceTextureClient::dequeueBuffer");
    Mutex::Autolock lock(mMutex);
    int buf = -1;
    status_t result = mSurfaceTexture->dequeueBuffer(&buf, mReqWidth, mReqHeight,
            mReqFormat, mReqUsage);
    if (result < 0) {
        LOGV("dequeueBuffer: ISurfaceTexture::dequeueBuffer(%d, %d, %d, %d)"
             "failed: %d", mReqWidth, mReqHeight, mReqFormat, mReqUsage,
             result);
        return result;
    }
    sp<GraphicBuffer>& gbuf(mSlots[buf]);
    if (result & ISurfaceTexture::RELEASE_ALL_BUFFERS) {
        freeAllBuffers();
    }
 
    if ((result & ISurfaceTexture::BUFFER_NEEDS_REALLOCATION) || gbuf == 0) {
        result = mSurfaceTexture->requestBuffer(buf, &gbuf);
        if (result != NO_ERROR) {
            LOGE("dequeueBuffer: ISurfaceTexture::requestBuffer failed: %d",
                    result);
            return result;
        }
    }
    *buffer = gbuf.get();
    return OK;
}
<style type="text/css"> .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; }</style>
int SurfaceTextureClient::queueBuffer(android_native_buffer_t* buffer) {
    LOGV("SurfaceTextureClient::queueBuffer");
    Mutex::Autolock lock(mMutex);
    int64_t timestamp;
    if (mTimestamp == NATIVE_WINDOW_TIMESTAMP_AUTO) {
        timestamp = systemTime(SYSTEM_TIME_MONOTONIC);
        LOGV("SurfaceTextureClient::queueBuffer making up timestamp: %.2f ms",
             timestamp / 1000000.f);
    } else {
        timestamp = mTimestamp;
    }
    int i = getSlotFromBufferLocked(buffer);
    if (i < 0) {
        return i;
    }
    status_t err = mSurfaceTexture->queueBuffer(i, timestamp,
            &mDefaultWidth, &mDefaultHeight, &mTransformHint);
    if (err != OK)  {
        LOGE("queueBuffer: error queuing buffer to SurfaceTexture, %d", err);
    }
    return err;
}
<style type="text/css"> .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; }</style>

我们可以看到,SurfaceTexture会利用持有的ISurfaceTextureClient接口去得到或入队GraphicBuffer。

我们前面也说过,Surface是一个ANativeWindow,而ANatvieWidnow是一个EGL可以操作的窗口,所以除了在java层显式调用lockCanvas方法可以操作Surface外,openGL ES通过EGL也能操作Surface。

 

5. GraphicBuffer

在具体分析之前,我们先来看一个ANativeWindow的dequeueBuffer以及queueBuffer原型:

    int     (*dequeueBuffer)(struct ANativeWindow* window,
                struct ANativeWindowBuffer** buffer);
 
    int     (*queueBuffer)(struct ANativeWindow* window,
                struct ANativeWindowBuffer* buffer);
 
ANativeWindow我们前面说过了,ANativeWindowBuffer我们还没有提及。我们本节讲的GraphicBuffer正是一个ANativeWindowBuffer,从前面我们看到,GraphicBuffer是需要通过ISurfaceTexture接口在客户端及服务端来传递的,所以其需要实现Flattenable接口。
GraphicBuffer实际的存储空间其实是在ashmem上的,具体是gralloc模块来完成分配的,然后映射到应用程序的进程地址空间。因为这部分涉及到底层设备部分,这里就不在分析。
<style type="text/css"> .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; }</style>

 

参考文章:

http://www.linuxidc.com/Linux/2012-03/55898p7.htm