I slowly continue with learning of OCAML – as a training project I work on simplified Map-Reduce framework (utilizing Core and Async libraries). Here I had a need to plug a selectable code (map reduce algorithm) to main program. Ocaml provides Dynlink library, which can dynamically link either byte-code or native object/library to running program. This can be utilize to create simple plugin framework as explained below.
Key function is
Dynlink.loadfile, which can load .cmo/cma (bytecode) or .cmxs file (native code) from given file. Loaded modules cannot be referenced from main program, so they must register themselves in defined extension points in main program during their load, where all code in plugin modules is executed. Key insight here (which is not well documented) is that loaded modules can reference only fully evaluated modules in main program – e.g. they cannot reference module, which is loading them.
In combination with first-class modules, we can build a plugin system with well defined interfaces. Below is simplified example how it may look like:
module type PLUG = sig val hello: unit -> string end let p = ref None let get_plugin () : (module PLUG) = match !p with | Some s -> s | None -> failwith "No plugin loaded"
Here we define interface
PLUG of the plugin, variable
p which will hold refrerence to plugin implementation (as first class module option) and function
open Dtest_ep let load_plug fname = let fname = Dynlink.adapt_filename fname in if Sys.file_exists fname then try Dynlink.loadfile fname with | (Dynlink.Error err) as e -> print_endline ("ERROR loading plugin: " ^ (Dynlink.error_message err) ); raise e | _ -> failwith "Unknow error while loading plugin" else failwith "Plugin file does not exist" let () = load_plug "/home/ivan/workspace/mapred/tests/_build/dtest_plug.cmo"; let module M = (val get_plugin () : PLUG) in print_endline (M.hello ())
This is main program – it contain function to load plugin and in main code uses plugin as module
open Dtest_ep module M:PLUG = struct let hello () = "Hello, World!" end let () = p := Some (module M:PLUG)
And this file is implementation of the plugin.
We can compile with:
ocamlbuild -lib dynlink dtest_plug.cmo dtest.byte ocamlbuild -lib dynlink dtest_plug.cmxs dtest.native
Plugin should be compiled against same version of interfaces as main program.
External modules in plugins
Plugin can use other external modules, but then it gets bit more tricky, if these modules not linked in main program – you’ll get ‘Unknown Global X’ error during dynamic linking of the plugin.
Solution is either to force module(s) to be linked in main program (
let _ = X.y ) or load required libraries with
Dynlink.loadfile (dynamically loaded modules will be visible from the plugin, but not to main program). Other option is to pack plugin with dependent object into one library file – I have not tested it, but I assume this should work.
I experienced Interesting behaviour when plugin ligrary contains same module as main program. In this case module from plugin library is loaded and linked to the plugin and hides module from main program.
For our sample above it means, if plugin will be packed in .cma file with Dtest_ep module object, plugin will not be able to register itself, because Dtest_ep from main program will be hidden from it.