DevelopingPlugins < System

---+ Developing Plugins
%TOC%
The usual way Foswiki is extended is by writing a _Plugin_. Plugins extend Foswiki by providing functions that 'listen' to events in the Foswiki core, and handling these events. These functions are called "Plugin Handlers" and they are described in depth in [[%SCRIPTURL{view}%/%SYSTEMWEB%/PerlDoc?module=Foswiki::Plugins::EmptyPlugin][EmptyPlugin]] ( =lib/Foswiki/Plugins/EmptyPlugin.pm= ).

---++ The 3048m view of how Foswiki works

Foswiki is a web application that runs inside a web server. When the web server receives a request that it recognises as being for Foswiki, it calls one of the perl scripts in the Foswiki =bin= directory. Each of the scripts has a specific function, as described in [[command and CGI scripts]].

The scripts are responsible for interpreting the parameters passed in the request, and generating a response that is sent back to the browser, usually in the form of an HTML page.

Foswiki contains three _engines_ that are used by the scripts; the _template engine_, the _macro engine_, and the _TML engine_.

   1 The *template engine* reads predefined templates from files on the server. These templates contain directives that are expanded by the engine to create the output HTML skeleton. One of these directives expands to the topic text.
   1 The *macro engine* then expands the [[macros]] in the skeleton. This is also where macros registered by plugins are expanded.
      * Macros, including those registered by plugins, are processed in a strict left-right-inside-out processing order. See [[macros]] for more details.
      * Macros include things like searches, so this is usually the slowest part of generating a page. 
   1 The *TML (Topic Markup Language) engine* now processes the expanded text, looking for TML constructs such as bulleted lists and tables. It generates HTML for these constructs.

Once all the engines have run, the output is sent to the browser.

There are several ways plugins can interact with this process.
   1 They can *register macros* that are expanded by the macro engine. This is the simplest kind of plugin.
   1 The can interact with various points in the rendering pipeline by implementing *handlers* (callbacks).
   1 They can *register REST handlers* that are invoked via the =rest= script to support some form of transaction outside those supported by the standard scripts.

---++ APIs available to Extensions
To be robust, extensions must avoid using any unpublished functionality from the Foswiki core. The following perl packages give access to features for extension authors. These APIs are not just for Plugins, they can be used in any type of extension. Click on the name of the package to see the full documentation.
   * =[[%SCRIPTURL{view}%/%SYSTEMWEB%/PerlDoc?module=Foswiki::Func][Foswiki::Func]]= - this is the package you will use most. This package exposes a lot of core functionality in a way that is friendly to extension writers. If you find that there are two ways of doing something - a =Foswiki::Func= way, and another call to one of the packages below, then the =Foswiki::Func= way is almost always the right way.
   * =[[%SCRIPTURL{view}%/%SYSTEMWEB%/PerlDoc?module=Foswiki::Meta][Foswiki::Meta]]= - topic and web meta-data. Certain =Foswiki::Func= methods, and some plugin handlers, are passed (or return) objects of this type. Almost all of the methods of =Foswiki::Meta= have analagous methods in =Foswiki::Func= - in general you should call the =Foswiki::Func= methods in preference to calling =Foswiki::Meta= methods directly.
   * =[[%SCRIPTURL{view}%/%SYSTEMWEB%/PerlDoc?module=Foswiki::OopsException][Foswiki::OopsException]]= - special exception for invoking the 'oops' script
   * =[[%SCRIPTURL{view}%/%SYSTEMWEB%/PerlDoc?module=Foswiki::AccessControlException][Foswiki::AccessControlException]]= - access control exception
   * =[[%SCRIPTURL{view}%/%SYSTEMWEB%/PerlDoc?module=Foswiki::Attrs][Foswiki::Attrs]]= - parser and storage object for macro parameters
   * =[[%SCRIPTURL{view}%/%SYSTEMWEB%/PerlDoc?module=Foswiki::Time][Foswiki::Time]]= - time parsing and formatting
   * =[[%SCRIPTURL{view}%/%SYSTEMWEB%/PerlDoc?module=Foswiki::Sandbox][Foswiki::Sandbox]]= - safe server-side program execution, used for calling external programs.
   * Iterators - these are classes that implement the =[[%SCRIPTURL{view}%/%SYSTEMWEB%/PerlDoc?module=Foswiki::Iterator][Foswiki::Iterator]]= specification
      * =[[%SCRIPTURL{view}%/%SYSTEMWEB%/PerlDoc?module=Foswiki::ListIterator][Foswiki::ListIterator]]= - utility class for iterator objects that iterate over list contents
      * =[[%SCRIPTURL{view}%/%SYSTEMWEB%/PerlDoc?module=Foswiki::LineIterator][Foswiki::LineIterator]]= - utility class for iterator objects that iterate over lines in a block of text
      * =[[%SCRIPTURL{view}%/%SYSTEMWEB%/PerlDoc?module=Foswiki::AggregateIterator][Foswiki::AggregateIterator]]= - utility class for iterator objects that aggregate other iterators into a single iteration
In addition the following global variables may be referred to:
   * =$Foswiki::Plugins::VERSION= - plugin handler API version number
   * =$Foswiki::Plugins::SESSION= - reference to =Foswiki= singleton object
   * =$Foswiki::cfg= - reference to configuration hash
   * =$Foswiki::regex= - see [[#StReEx][Standard Regular Expressions]], below
<blockquote class="foswikiHelp">%I% Foswiki:Development.GettingStarted  is the starting point for more comprehensive documentation on developing for Foswiki.</blockquote>

*Note:* the APIs are available to all extensions, but rely on a
=Foswiki= singleton object having been created before the APIs can be used.
This will only be a problem if you are writing an extension that doesn't
use the standard initialisation sequence.

#StReEx
---+++ Standard Regular Expressions
A number of standard regular expressions are available for use in extensions, in the =$Foswiki::regex= hash. these regular expressions are precompiled in an
<nop>I18N-compatible manner. The
following are guaranteed to be present. Others may exist, but their use
is unsupported and they may be removed in future Foswiki versions.

In the table below, the expression marked type 'String' are intended for
use within character classes (i.e. for use within square brackets inside
a regular expression), for example:
<verbatim class="perl">
   my $isCapitalizedWord =
     ( $s =~ /[$Foswiki::regex{upperAlpha}][$Foswiki::regex{mixedAlpha}]+/ );
</verbatim>
Those expressions marked type 'RE' are precompiled regular expressions that can be used outside square brackets. For example:
<verbatim class="perl">
   my $isWebName = ( $s =~ m/$Foswiki::regex{webNameRegex}/ );
</verbatim>

| *Name*         | *Matches*                        | *Type* |
| upperAlpha     | Upper case characters            | String |
| upperAlphaNum  | Upper case characters and digits | String |
| lowerAlpha     | Lower case characters            | String |
| lowerAlphaNum  | Lower case characters and digits | String |
| numeric        | Digits                           | String |
| mixedAlpha     | Alphabetic characters            | String |
| mixedAlphaNum  | Alphanumeric characters          | String |
| wikiWordRegex  | WikiWords                        | RE |
| webNameRegex   | User web names                   | RE |
| topicNameRegex | Topic names                      | RE |
| anchorRegex    | #AnchorNames                     | RE |
| abbrevRegex    | Abbreviations/Acronyms e.g. GOV, IRS | RE |
| emailAddrRegex | email@address.com                | RE |
| tagNameRegex   | Standard macro names e.g. %<nop>THIS_BIT% (THIS_BIT only) | RE |

#CreatePlugins
---++ Creating New Plugins

With a reasonable knowledge of the Perl scripting language, you can create new plugins or modify and extend existing ones. 

---+++ Anatomy of a Plugin

A (very) basic Foswiki plugin consists of two files:

   * a Perl module, e.g. =lib/Foswiki/Plugins/MyFirstPlugin.pm=
   * a documentation topic, e.g. =MyFirstPlugin.txt=

The Perl module can invoke other, non-Foswiki, elements, like other Perl modules (including other plugins), graphics, external applications, or just about anything else that Perl can call.

The plugin API handles the details of connecting your Perl module with the Foswiki core.

_The Foswiki:Extensions.BuildContrib module provides a lot of support for plugins development, including a plugin creator, automatic publishing support, and automatic installation script writer. If you plan on writing more than one plugin, you probably need it_.

---+++ Creating the Perl Module 

Copy file =lib/Foswiki/Plugins/EmptyPlugin.pm= to =&lt;name&gt;Plugin.pm=. The EmptyPlugin does nothing, but it contains all the information you need to create you own custom plugin.

#CreatePluginTopic
---+++ Writing the Documentation Topic

The plugin documentation topic contains usage instructions and version details. (The doc topic is also included _in_ the [[#CreatePluginPackage][distribution package]].) To create a documentation topic:

   1. *Copy* the plugin topic template from [[%SYSTEMWEB%.EmptyPlugin?raw=on][EmptyPlugin]]
   1. *Customize* your plugin topic.
      * Important: In case you plan to publish your plugin on Foswiki.org, use Interwiki names for author names and links to Foswiki.org topics, such as Foswiki:Main/%WIKINAME%. This is important because links should work properly in a plugin topic installed on any Foswiki, not just on Foswiki.org.
   1. *Save* your topic, for use in [[#CreatePluginPackage][packaging]] and [[#PublishPlugin][publishing]] your plugin.

<blockquote style="background-color:#f5f5f5">
*OUTLINE: Doc Topic Contents* <br />
Check the plugins web on Foswiki.org for the latest plugin doc topic template. Here's a quick overview of what's covered:

*Syntax Rules:* &lt;<i>Describe any special text formatting that will be rendered.</i>&gt;" 

*Example:* &lt;<i>Include an example of the plugin in action. Possibly include a static HTML version of the example to compare if the installation was a success!</i>&gt;"

*Plugin Settings:* &lt;<i>Description and settings for custom plugin settings, and those required by Foswiki.</i>&gt;"

   * *Plugins Preferences* &lt;<i>If user settings are needed, link to [[%SYSTEMWEB%.PreferenceSettings][preference settings]] and explain the role of the plugin name prefix</i>

*Plugin Installation Instructions:* &lt;<i>Step-by-step set-up guide, user help, whatever it takes to install and run, goes here.</i>&gt;"

*Plugin Info:* &lt;<i>Version, credits, history, requirements - entered in a form, displayed as a table. Both are automatically generated when you create or edit a page in the Foswiki:Extensions web.</i>&gt;
</blockquote>

#CreatePluginPackage
---+++ Packaging for Distribution

The Foswiki:Extensions.BuildContrib is a powerful build environment that is used by the Foswiki project to build Foswiki itself, as well as many of the plugins. You don't *have* to use it, but it is highly recommended!

If you don't want to (or can't) use the !BuildContrib, then a minimum plugin release consists of a Perl module with a WikiName that ends in =Plugin=, ex: =MyFirstPlugin.pm=, and a documentation page with the same name(=MyFirstPlugin.txt=).

   1. Distribute the plugin files in a directory structure that mirrors Foswiki. If your plugin uses additional files, include them all:
      * =lib/Foswiki/Plugins/MyFirstPlugin.pm=
      * =data/Foswiki/MyFirstPlugin.txt=
      * =pub/Foswiki/MyFirstPlugin/uparrow.gif= [a required graphic]
   2. Create a zip archive with the plugin name (=MyFirstPlugin.zip=) and add the entire directory structure from Step 1. The archive should look like this:
      * =lib/Foswiki/Plugins/MyFirstPlugin.pm=
      * =data/Foswiki/MyFirstPlugin.txt=
      * =pub/Foswiki/MyFirstPlugin/uparrow.gif=

#PublishPlugin
---+++ Publishing for Public Use

You can release your tested, packaged plugin to the Foswiki community through the Foswiki:Extensions web. All plugins submitted to Foswiki.org are available for public download and further development.

Publish your plugin by following these steps:
   1. *Post* the plugin documentation topic to the Foswiki:Extensions web
   1. *Attach* the distribution zip file(s) to the topic, eg:
      =MyFirstPlugin.zip=
   1. Add a user support hub by visiting Foswiki:Support.CreateNewSupportHub
   1. Optionally, check in the sources to the Foswiki subversion repository
      (see Foswiki:Development.HowToStartExtensionDevelopmentInSubversion)

%N% Once you have done the above steps once, you can use the !BuildContrib to upload updates to your plugin.

Thank you very much for sharing your plugin with the Foswiki community :-)

#FastPluginHints
---++ Hints on Writing Fast Plugins

   * Delay initialization as late as possible. For example, if your plugin is a simple syntax processor, you might delay loading extra Perl modules until you actually see the syntax in the text.
      * For example, use an =eval= block like this:%BR%
        =eval { require IPC::Run }= %BR%
        =return "&lt;font color=\"red\"&gt;SamplePlugin: Can't load required modules ($@)&lt;/font&gt;" if $@;=
   * Keep the main plugin package as small as possible; create other packages that are loaded if and only if they are used. For example, create sub-packages of !BathPlugin in =lib/Foswiki/Plugins/BathPlugin/=.
   * Avoid using preferences in the plugin topic; set =$NO_PREFS_IN_TOPIC= if you possibly can, as that will stop Foswiki from reading the plugin topic for every page. Use [[#ConfigSpec][Config.spec]] instead. 
   * Use registered tag handlers

---++ Security

   * Badly written plugins can open security holes in Foswiki. This is especially true if care isn't taken to prevent execution of arbitrary commands on the server.
   * Don't allow sensitive configuration data to be edited by users. Use the =%Foswiki::cfg= hash for configuration options. Don't ask installers to edit topics in the %SYSTEMWEB% web.
      * [[#ConfigSpec][Integrating with <code>configure</code>]] describes the steps
      * Foswiki:Extensions.MailInContrib has an example
      * Foswiki:Extensions.BuildContrib can help you with this
   * Make sure that all user input is checked and validated. Be especially careful to filter characters that might be used in perl string interpolation.
   * Avoid =eval=, and if you must use it make sure you sanitise parameters
   * Always use the Foswiki::sandbox to execute commands. Never use backtick or qx//.
   * Use =Foswiki::Func::checkAccessPermission= to check the access rights of the current user.
   * Always audit the plugins you install, and make sure you are happy with the level of security provided. While every effort is made to monitor plugin authors activities, at the end of the day they are uncontrolled user contributions.

#RecommendedStorageOfPluginData
---++ Recommended Storage of Plugin Specific Data

Plugins sometimes need to store data. This can be plugin internal data such as cache data, or data generated for browser consumption such as images. Plugins should store data using [[%SCRIPTURL{view}%/%SYSTEMWEB%/PerlDoc?module=Foswiki::Func][Foswiki::Func]] functions that support saving and loading of topics and attachments.

---+++ Plugin Internal Data

You can create a plugin "work area" using the =Foswiki::Func::getWorkArea()= function, which gives you a persistent directory where you can store data files. By default they will not be web accessible. The directory is guaranteed to exist, and to be writable by the webserver user. For convenience, =Foswiki::Func::storeFile()= and =Foswiki::Func::readFile()= are provided to persistently store and retrieve simple data in this area.

---+++ Web Accessible Data

The internal data area is not normally made web-accessible for security reasons. If yoou want to store web accessible data, for example generated images, then you should use Foswiki's attachment mechanisms.

*Topic-specific data* such as generated images can be stored in the topic's attachment area, which is web accessible. Use the =Foswiki::Func::saveAttachment()= function to store the data.

Recommendation for file name:
   * Prefix the filename with an underscore (the leading underscore avoids a name clash with files attached to the same topic)
   * Identify where the attachment originated from, typically by including the plugin name in the file name
   * Use only alphanumeric characters, underscores, dashes and periods to avoid platform dependency issues and URL issues
   * Example: =_GaugePlugin_img123.gif=
Such auto-generated attachments han be hidden from users by setting the 'h' attribute in the attachment attributes.

*Web specific data* should be stored in the attachment area of a topic in the web that you specify for the purpose, e.g. Web.<nop>BathPlugPictures. Use the =Foswiki::Func::saveAttachment()= function to store the data in this topic.

#ConfigSpec
---++ Integrating with <code>configure</code>
Some extensions have setup requirements that are best integrated into =configure= rather than trying to use [[%SYSTEMWEB%.PreferenceSettings][preference settings]]. These extensions use =Config.spec= files to publish their configuration requirements.

=Config.spec= files are read during configuration. Once a =Config.spec= has defined a configuration item, it is available for edit through the standard =configure= interface. =Config.spec= files are stored in the 'plugin directory' e.g. =lib/Foswiki/Plugins/BathPlugin/Config.spec=.

---+++ Structure of a <code>Config.spec</code> file
The =Config.spec= file for a plugin starts with a line that declares what section the configuration should appear in. The standard for all extensions is:
<verbatim class="tml">
# ---+ Extensions
</verbatim>
Next we have a sub-heading for the configuration specific to this extension,
and the actual configuration options:
<verbatim class="tml">
# ----++ BathPlugin
# This plugin senses the level of water in your bath, and ensures the plug
# is not removed while the water is still warm.
</verbatim> 
This is followed by one or more configuration items. Each configuration item has a _type_, a _description_ and a _default_. For example:
<verbatim class="perl">
# **SELECT Plastic,Rubber,Metal**
# Select the plug type
$Foswiki::cfg{BathPlugin}{PlugType} = 'Plastic';

# **NUMBER**
# Enter the chain length in cm
$Foswiki::cfg{BathPlugin}{ChainLength} = '30';

# **BOOLEAN EXPERT**
# Turn this option off to disable the water temperature alarm
$Foswiki::cfg{BathPlugin}{TempSensorEnabled} = '1';
</verbatim>
The type (e.g. =**SELECT**= ) tells =configure= to how to prompt for the value. It also tells configure how to do some basic checking on the value you actually enter. All the comments between the type and the configuration item are taken as part of the description. The configuration item itself defines the default value for the configuration item. The above spec defines the configuration items =$Foswiki::cfg{BathPlugin}{PlugType}=, =$Foswiki::cfg{BathPlugin}{ChainLength}=, and =$Foswiki::cfg{BathPlugin}{TempSensorEnabled}= for use in your plugin. For example,
<verbatim class="perl">
if( $Foswiki::cfg{BathPlugin}{TempSensorEnabled} && $curTemperature > 50 ) {
    die "The bathwater is too hot for comfort";
}
</verbatim>
You can use other =$Foswiki::cfg= values in other settings,
but you must be sure they are only evaluated under program control, and
not when this file is parsed by perl. For example:
<verbatim class="perl">
$Foswiki::cfg{BathPlugin}{MyBath} = "$Foswiki::cfg{PubDir}/enamel.gif"; # BAD
# Perl will interpolate variables in double-quotes, so $Foswiki::cfg{PubDir}
# will be evaluated at configuration time, which will make reconfiguration
# difficult.

$Foswiki::cfg{BathPlugin}{MyBath} = '$Foswiki::cfg{PubDir}/enamel.gif'; # GOOD
# The single quotes make sure $Foswiki::cfg{PubDir} will only be evaluated
# at run-time.
</verbatim>

The =Config.spec= file is read by configure, and =configure= then writes =LocalSite.cfg= with the values chosen by the local site admin.

A range of types are available for use in =Config.spec= files:

| =BOOLEAN= | A true/false value, represented as a checkbox |
| =COMMAND= _length_ | A shell command |
| =LANGUAGE= | A language (selected from ={LocalesDir}= |
| =NUMBER= | A number |
| =OCTAL= | An octal number |
| =PASSWORD= _length_ | A password (input is hidden) |
| =PATH= _length_ | A file path |
| =PERL= | A simplified perl data structure, consisting of arrays, hashes and scalar values |
| =REGEX= _length_ | A perl regular expression |
| =SELECT= _choices_ | Pick one of a range of choices |
| <code>SELECTCLASS</code>&nbsp;<em>package-specifier</em> | Select a perl package (class) e.g. =SELECTCLASS Foswiki::Plugins::BathPlugin::*Plug= lets the user select between all packages with names ending in =Plug=, =Foswiki::Plugins::BathPlugin::RubberPlug=, =Foswiki::Plugins::BathPlugin::BrassPlug= etc. |
| =STRING= _length_ | A string |
| =URL= _length_ | A url |
| =URLPATH= _length_ | A relative URL path |

All types can be followed by a comma-separated list of _attributes_.
| =EXPERT= | means this an expert option |
| =M= | means the setting is mandatory (may not be empty) |
| =H= | means the option is not visible in =configure= |
| =5x80= | means "use a 5 row, 80 column textarea". Can be used with any text entry field type, such as STRING, COMMAND, PERL etc. |

See =lib/Foswiki.spec= for many more examples.

=Config.spec= files are also used for other (non-plugin) extensions. in this case they are stored under the =Contrib= directory instead of the =Plugins= directory.

#LinkingToConfigure
---+++ Linking to =configure=

You can link to your =configure= settings by using the following:

<verbatim class="tml">
[[%SCRIPTURL{"configure"}%/#BathPlugin$Extensions][configure]]
</verbatim>

Replace =BathPlugin= with the name of your extension.

#MaintainPlugins
---++ Maintaining Plugins

---+++ Discussions and Feedback on Plugins

Usually published plugins have a support hub in the Support web on Foswiki.org.Support hubs have links to where to discuss feature enhancements and give feedback to the developer and user communities.

---+++ Maintaining Compatibility with Earlier Foswiki Versions

The plugin interface ([[%SCRIPTURL{view}%/%SYSTEMWEB%/PerlDoc?module=Foswiki::Func][Foswiki::Func]] functions and plugin handlers) evolve over time. Foswiki introduces new API functions to address the needs of plugin authors. Plugins using unofficial Foswiki internal functions may no longer work on a Foswiki upgrade.

Organizations typically do not upgrade to the latest Foswiki for many months. However, many administrators still would like to install the latest versions of a plugin on their older Foswiki installation. This need is fulfilled if plugins are maintained in a compatible manner. 

<blockquote class="foswikiHelp"> *%T% Tip:* Plugins can be written to be compatible with older and newer Foswiki releases. This can be done also for plugins using unofficial Foswiki internal functions of an earlier release that no longer work on the latest Foswiki codebase. 
Here is an example; the Foswiki:Support.PluginsSupplement has more details.</blockquote>

<verbatim class="perl">
    if( $Foswiki::Plugins::VERSION >= 1.1 ) {
        @webs = Foswiki::Func::getListOfWebs( 'user,public' );
    } else {
        @webs = Foswiki::Func::getPublicWebList( );
    }
</verbatim>

---+++ Handling deprecated functions

From time-to-time, the Foswiki developers will add new functions to the interface (either to [[%SCRIPTURL{view}%/%SYSTEMWEB%/PerlDoc?module=Foswiki::Func][Foswiki::Func]], or new handlers). Sometimes these improvements mean that old functions have to be deprecated to keep the code manageable. When this happens, the deprecated functions will be supported in the interface for at least one more Foswiki release, and probably longer, though this cannot be guaranteed.

When a plugin defines deprecated handlers, a warning will be shown in the list generated by %<nop>FAILEDPLUGINS%. Admins who see these warnings should check Foswiki.org and if necessary, contact the plugin author, for an updated version of the plugin.

Updated plugins may still need to define deprecated handlers for compatibility with old Foswiki versions. In this case, the plugin package that defines old handlers can suppress the warnings in %<nop>FAILEDPLUGINS%.

This is done by defining a map from the handler name to the =Foswiki::Plugins= version _in which the handler was first deprecated_. For example, if we need to define the =endRenderingHandler= for compatibility with =Foswiki::Plugins= versions before 1.1, we would add this to the plugin:

<verbatim class="perl">
package Foswiki::Plugins::SinkPlugin;
use vars qw( %FoswikiCompatibility );
$FoswikiCompatibility{endRenderingHandler} = 1.1;
</verbatim>
If the currently-running Foswiki version is 1.1 _or later_, then the _handler will not be called_ and _the warning will not be issued_. Foswiki with versions of =Foswiki::Plugins= before 1.1 will still call the handler as required.

---+++ TWiki<sup>&reg;</sup> Plugins
Most plugins written for TWiki can also be run in Foswiki, by installing the !TWikiCompatibilityPlugin. See Foswiki:Extensions.TWikiCompatibilityPlugin for more information.
<!-- %JQREQUIRE{"chili"}% -->
This topic: System > WebHome > ReferenceManual > DeveloperDocumentationCategory > DevelopingPlugins
Topic revision: revision 1 (raw view)
Copyright &© by the contributing authors. All material on this site is the property of the contributing authors.
Ideas, requests, problems regarding BACCHUS Wiki? Send feedback