This document describes the steps involved in providing support for asynchronous access of HSF data in your application. It assumes familiarity with HOOPS/3dGS and HOOPS/Stream and uses the HOOPS/ActiveX Control as a reference. While several pieces of code may include Windows or MFC specific functions, the majority of the code is platform independent, and most of the effort in supporting asynchronous data access within your application involves either:

1. finding comparable functions to call in place of the MFC-specific functions (for example, the function that creates a new thread)

or

2. placing certain pieces of code within functions that are comparable to MFC functions (for example, the function that is called when the window needs to be repainted. With X11, this is an ‘Expose’ event, with MFC this is a WM_PAINT event.)

While reading each section of this document, it is highly recommended that you carefully read through the associated HOOPS/ActiveX source code and comments located in the CHoopsControl, StreamingThread and CHoopsDataLoader .cpp/.h files, which are all part of the HOOPS/ActiveX project located in the /dev_tools/hoops_activex/source directory of your HOOPS/3dAF installation.

Below we describe how asynchronous data access is handled in a ActiveX control derived from the HOOPS/ActiveX base classes. The work done here is analogous to the work you would need to do within your own application or plug-in.

1. Downloading of a HOOPS Stream File (HSF) is initiated by setting the ‘Filename’ property of the HOOPS/ActiveX-based control. This causes an internal HOOPS/ActiveX function to load the URL which references the HSF file (i.e. http://www.mycompany.com/part.hsf). Specifically, a special MFC object is created which automatically connects to the data download facilities of Microsoft Internet Explorer, and a method of this object is automatically called by the framework when data is available (meaning, a new buffer of data has been downloaded from the Internet). This special MFC object automatically runs on a separate thread. The key point here is that the HSF data is supplied to the HOOPS/ActiveX control via a callback mechanism. The framework determines when data is available and how much there is, and presents it to us. When a buffer of data is presented, it is placed in a shared linked-list of buffers.
   
2. A separate thread is created, which continually reads buffers from the linked list and sends a message to the main thread indicating that a new buffer is ready for processing.
   
3. The main thread handles these ‘new_buffer’ messages, and uses HOOPS/Stream to parse and insert the data into the HOOPS/3dGS scene-graph. After each new buffer of HSF data has been parsed and inserted, a request is made to HOOPS/3dGS to update the scene. HOOPS/3dGS has built-in support for incrementally adding the new geometry to the scene without having to completely redraw it.

The following diagram provides a visual overview of the HOOPS/ActiveX Control data-streaming architecture.

Detailed Function Call Sequence

Below we review the detailed sequence of calls in the HOOPS 3D Stream Control. While following the calling sequence, it is recommended that you review the associated code/comments in the CHoopsControl and CHoopsDataLoader .cpp/.h files.

1. Application Initialization
2. The main object in the HOOPS 3D Stream Control is the CHoopsControl object. All initialization of the streaming related variables occurs in the CHoopsControl constructor. These include a critical section called ‘buffer_list_lock’ to deal with the linked list of HSF data buffers which will be shared by two different threads.
3. File access initiated

CHoopsControl also provides a method called LoadFromUrl, which would be called by the custom ActiveX controlled object (derived from CHoopsControl) when a HSF needs to be downloaded. (The custom ActiveX control would typically call CHoopsControl::LoadFromUrl when its ‘Filename’ property has changed). You will need to identify an analogous function in your GUI environment, which provides a request to download a file.

CHoopsControl::LoadFromUrl creates a custom asynchronous data loader callback object called CHoopsDataLoader and then calls CHoopsDataLoader::LoadFromUrl

CHoopsDataLoader::LoadFromUrl makes an MFC-specific call (AfxBeginThread) to spawn the new thread which will be used to read HSF buffers from a shared linked-list. You will need to identify an analogous function to create a new thread in your GUI/programming environment.

The thread executes the function called HStreamFileReadingThread. It also creates the MFC-specific asynchronous data download object called CHoopsDataUrl, and calls CHoopsDataUrl->Open to actually begin the download process. You will need to identify an analogous data download mechanism in your GUI/programming environment.

1. Processing callbacks from the File Reading Thread

   The framework automatically calls the following CHoopsDataLoader methods:

OnDownloadStart() - indicates that downloaded has begun; the version mismatch variable needs to be re-initialized here.

OnDownloadStop() - indicates that downloading is complete; no extra stream-related code is necessary here

OnDownloadProgress() - informs us of how much data has been downloaded so far; no extra stream-related code is necessary here, though code could be added to track the progress and put up a UI progress bar

OnDownloadDataAvailable(); - informs us that another buffer of data has been downloaded and returns it. If necessary, we break the buffer into smaller buffers so that the max size equals BUFFERSIZE. The entire file could come in as a single buffer (for example when the file is loaded from the local cache) and then a single buffer would be posted to the main thread for processing which would prevent asynchronous streaming. Consequently, we always want to break the buffers up into smaller ones that won’t take too long for the main thread to process. You should be able to use the same buffer/linked-list code, but again, you need to use the facilities of your GUI/programming environment to obtain buffers of data that are coming over the Internet. The MFC-specific object that is handling the download is powerful, in that it runs on a separate thread, and calls us back when data is available. You should ideally utilize (or create) a similar mechanism in your programming environment.

The buffers are placed into the linked list containing StreamBuffer objects that is to be shared with the separate buffer-reading thread. (Recall that this thread had previously been spawned, and is already looking for new buffers.) Our main thread had to make a copy of the new buffer that came into the CHoopsDataLoader object (pBuffer), so rather than copying again, each StreamBuffer object simply points to offsets of the pBuffer array. The last StreamBuffer object’s ‘deleteme’ member contains a pointer to the original pBuffer array so that we are able to delete it when we’re done with processing all the buffers.

All list access code must reside within the ‘buffer_list_lock’ critical section.

1. The HStreamFileReadingThread function executes

   This separate thread goes into a ‘forever’ loop that looks at the shared linked list containing StreamBuffer objects. If it is non-null, it posts a message (CWnd::PostMessage) to the main thread which indicates that a new buffer needs to be processed (the main thread will use HOOPS/Stream to parse and insert the 2D/3D data into the HOOPS/3dGS scene-graph).

   Within the forever loop, the thread Sleeps if terminate is not true, and any of the following are true:

   1. The main thread has set the CHoopsControl ‘pause_streaming’ variable to true, which is a feature where the user can configure the HOOPS/ActiveX control to pause the streaming process if the user is clicking and dragging the mouse (interacting with the scene).
      
   2. The CHoopsControl ‘counter’ variable is > 0, which means that there is already a buffer pending on the main process.

      A message is only sent to the main thread if there aren’t more than a certain [user-controlled] number buffers pending on the main thread (in the HOOPS/ActiveX base control object, the number of pending buffers allowed is 1). This is important because if the HStreamFileReadingThread always posted ‘new_buffer’ messages as soon as it noticed new buffers, then we could have a situation where numerous messages might get posted very quickly (perhaps the file is local or the Internet connection is very fast), and then the main thread would spend a significant amount of time time processing the buffers. Processing the buffers which involves parsing them and inserting them into theHOOPS/3dGS scene-graph, causes the main application thread, which processes the application events, to take a secondary role resulting in the user being unable to nicely interact with the scene as data is being streamed in. So, the key to asynchronous streaming is to make sure that the main thread is able to devote attention to other events/messages (mouse move/click, etc…), and every so often devote attention to dealing with a new buffer of HSF data that needs to be added into the scene-graph and drawn.

      With the exception of the PostMessage function and the platform specific checks to make sure the GUI window has already been created, you should generally be able to utilize the HStreamFileReadingThread function as is. Locate the appropriate functions to use in place of PostMessage and IsWindow for your programming environment.

   3. The CHoopsControl object doesn’t have a valid GUI window handle yet. (If we processed a new HSF data buffer without a valid window, we’d get an error because processing includes parsing and instructing HOOPS/3dGS to incrementally add the data to the scene and draw it to the screen. You can’t draw if there’s not a window available yet.)

   We break out of the loop if the CHoopsControl ‘terminate’ variable (also shared by both the main thread and the buffer-reading thread) is true. This could happen if:

   1. The main thread has processed the last file buffer.
   2. The file was an incorrect version.
   3. The CHoopsControl object was destroyed and data downloading was aborted (the user clicked the ‘Back’ button within Internet Explorer, etc…)

   After breaking out of the ‘forever’ loop, the streaming thread needs to clean up any leftover buffers.

   As usual, all list access code must reside within the ‘buffer_list_lock’ critical section.

2. The main thread handles the messages indicating that there is a new HSF data buffer to be processed.

   This is handled in CHoopsControl::OnNewStreamFileBuffer. This function processes new HSF data buffer messages that have been posted by the separate buffer-reading thread. The HStreamFileToolkit::ParseBuffer method (which is part of HOOPS/Stream) is called. In review, this parses the buffer of HSF data, and inserts the corresponding geometry, attribute, and segment information into the HOOPS/3dGS scene-graph. The return value of ParseBuffer is checked for:

   1. TK_Version: this indicates a version mismatch, where the version of the HOOPS/Stream that is being used to perform HSF reading (parsing) is not officially compatible with the version of the HSF file. One option is to continue trying to read, because the version mismatch may not cause an error. However, if an error DOES occur after a TK_Version has been encountered, the odds are that the error is due to a versioning problem.
   2. TK_Error: the HSF data buffer could not be processed. Abort the entire reading process (and cause the separate buffer-reading thread to exit) by setting ‘terminate’ to true.
   3. TK_Complete: this was the last HSF buffer; file streaming is complete, so set ‘terminate’ to true.

   Finally, a request is made to HOOPS/3dGS to update the scene (which will cause any new scene-graph objects to be incrementally added) by calling m_pHoopsView->Update() This calls the HOOPS/MVO HBaseView object’s Update() function, which ultimately calls through to HOOPS/3dGS. We only want to update after each new buffer has been processed if the CHoopsControl UI window has already been mapped to the screen (GetFirstUpdate() returns true)

   Application Shutdown

   It is possible that the main CHoopsControl object was destroyed while asynchronous streaming was still occurring. Therefore, we need to:

   1. shut down and delete the ActiveX-specific CHoopsDataLoader object.
   2. set ‘terminate’ to true so that the reading thread exits.
   3. if m_bReadingComplete is not true (meaning the separate thread has not yet exited) then sleep until ‘terminate’ is set to false (by the separate thread).
   4. after we are finished sleeping (meaning the separate thread has exited), it is not safe to delete the ‘buffer_list_lock’ critical section.

We want the initial HOOPS/3dGS camera setting to be able to view the total extents of the scene (and not continually change as each new piece of the scene is streamed in). However, we of course don’t have the scene yet since it’s being streamed over!

HOOPS/Stream provides facilities to address this. It supports an opcode called TK_Bounding. After this opcode has been processed (parsed and mapped to appropriate HOOPS/3dGS objects), the HOOPS/3dGS scene graph will contain the scene bounding volume information that is necessary to fit the camera to the entire scene. This means that we register a custom TK_Bounding opcode handler with the HOOPS/Stream toollkit, overload its Execute method, and instruct HOOPS/3dAF to FitWorld within that method (which will fit the camera to the scene extents.)

We create a custom TK_Bounding opcode handler called TK_Custom_Bounding (defined in CHoopsControl.h) TK_Custom_Bounding::Execute is implemented in CHoopsControl.cpp.