Plugins

The Grouparoo Plugin System is how you can add & extend Grouparoo!

Installing a Plugin

In your Grouparoo deployment project, simply install the Plugin with NPM and enable it in your package.json file

1. npm install @grouparoo/awesome-plugin
2. Add the Plugin to your grouparoo -> plugins hash within package.json. When complete, your package.json using @grouparoo/awesome-plugin will look like:

{
  "author": "Grouparoo Inc <hello@grouparoo.com>",
  "name": "@grouparoo/example-app",
  "description": "A Grouparoo Example Application",
  "version": "0.1.0",
  "license": "UNLICENSED",
  "private": true,
  "dependencies": {
    "@grouparoo/core": "1.0.0",
    "@grouparoo/awesome-plugin": "1.0.0"
  },
  "scripts": {
    "start": "cd node_modules/@grouparoo/core && ./bin/start",
    "dev": "cd node_modules/@grouparoo/core && ./bin/dev"
  },
  "grouparoo": {
    "plugins": ["@grouparoo/awesome-plugin"] // <--- HERE
  }
}

Developing a Plugin

Mainly, Plugins add new Sources and Destinations to interact with new databases and APIs. However Plugins also can:

• Add API actions
• Add new API tasks/background jobs
• Add new initializers and middleware
• Interact with the database (via models)
• Add new CLI commands for the API

Setup

A Grouparoo Plugin is based on the Actionhero Plugin model (https://docs.actionherojs.com/tutorial-plugins.html), and we have many hooks provided via package.json elements which you can use in your Plugin.

Your directory structure for your Plugin should look like:

/
| / src
  | - actions
  | - tasks
  | - initializers
  | - bin
  | - components
|
| - package.json
| - ts.config.json

The best way to develop you Plugin is to create a parent Grouparoo project with a package.json (see "Installing a Plugin" above) which requires both @grouparoo/core and your new Plugin. From there, you can either mount your local Plugin as you develop it with tools like npm link or pnpm for more complex projects in a monorepo.

Below, we will enumerate the types of things your plugins can do:

• package.json options
• Initializers
• Grouparoo Apps & Connections
• Actions & Routes
• Define and use Settings
• Tasks
• UI Elements

package.json options

In your Plugin's package.json, we use a grouparoo section to list certain options:

{
  "name": "@grouparoo/awesome-plugin",
  "description": "a Grouparoo Plugin",
  "version": "1.0.0",
  "license": "UNLICENSED",
  "private": true,
  "engines": {
    "node": ">=12.0.0"
  },
  "scripts": {
    "prepare": "rm -rf dist && tsc --declaration",
    "test": "jest",
    "pretest": "npm run lint && npm run prepare",
    "lint": "prettier --check src/**/*.ts __tests__/**/*.ts"
  },
  "dependencies": {
    "something": "^5.10.0"
  },
  "peerDependencies": {
    "@grouparoo/core": "1.0.0",
    "actionhero": "^22.0.5"
  },
  "devDependencies": {
    "@grouparoo/core": "1.0.0",
    "actionhero": "^22.0.5",
    "@types/jest": "^25.1.4",
    "@types/node": "^13.9.0",
    "jest": "^25.1.0",
    "prettier": "^1.19.1",
    "ts-jest": "^25.2.1",
    "typescript": "^3.8.3"
  },
  "grouparoo": {
    // <-- HERE
    "env": {
      "api": ["APP_API_KEY"]
    },
    "serverInjection": ["/path/to/file.js"]
  }
}

grouparoo/env

• The api key defines the environment variables that your application needs to launch.
  • If these environment variables are not present, the Grouparoo Core application will display an error, but will still launch
• This example means that process.env.APP_API_KEY can be used within your Plugin (if the user provides it).

grouparoo/serverInjection

• This array of files will be required very early on in the server boot process (before actionhero and config).
• This is a great place to load a bug tracker or process monitor early on in the server's life cycle
• This file needs to export a default (non-async) function which will then be run at boot.
• If you are using Typescript, be sure to only include the compiled dist version of the file

For Example, here is how @grouparoo/new-relic is initialized:

// from `@grouparoo/new-relic/src/serverInjection.ts`

if (process.env.NEW_RELIC_LICENSE_KEY) {
  require("newrelic");
}

export default function main() {
  if (process.env.NEW_RELIC_LICENSE_KEY) {
    console.log("newrelic injected into application");
  }
}

Notes:

• Ensure that the version of Grouparoo you are targeting is in both peerDependencies and devDependencies. It should not be in dependencies.
• If you are creating initializers, actions, or tasks, actionhero should be in both peerDependencies and devDependencies. It should not be in dependencies.

Connections

Your Plugin can provide multiple types of Apps for use within Grouparoo. Grouparoo users can create multiple instances of each type of App. It can then create Connections to import and/or export data.

More information is available for each type:

• Import via sources
• Export via destinations

Initializers

An initializer is an Actionhero file which lives in /src/initializers. There are 3 lifecycle parts of an initializer: initialize, start, and stop in which you can run code.

Learn more about Actionhero initializers here.

Within initializers, you can register routes and create apps and connections for Grouparoo. If your Plugin is providing apps or connections you can use the plugin.registerPlugin method. See the simplified example below. You can see the full file from @grouparoo/postgres here

import { Initializer } from "actionhero";
import { plugin } from "@grouparoo/core";

import { test } from "./../lib/test";
import { columns } from "./../lib/columns";
import {
  recordPropertyRuleQueryOptions,
  buildRecordPropertyRuleQuery,
} from "./../lib/buildRecordQuery";
import { records } from "./../lib/records";
import { recordProperty } from "./../lib/recordProperty";
import { exportRecord } from "./../lib/exportRecord";

export class Plugins extends Initializer {
  constructor() {
    super();
    this.name = "@grouparoo/postgres";
  }

  async initialize() {
    plugin.registerPlugin({
      name: "@grouparoo/postgres",
      icon: "/public/@grouparoo/postgres/postgres.png",
      apps: [
        {
          name: "postgres",
          options: [
            { key: "host", required: false, description: "the postgres host" },
            { key: "port", required: false, description: "the postgres port" },
            { key: "user", required: false, description: "the postgres user" },
            // ...
          ],
          test: test,
          recordPropertyRuleQueryOptions,
          buildRecordPropertyRuleQuery,
        },
      ],
      connections: [
        {
          name: "postgres-import",
          direction: "import",
          description: "import or update records from a postgres database",
          app: ["postgres"],
          options: [
            { key: "table", required: true, description: "the table to scan" },
            // ...
          ],
          methods: {
            records,
            recordProperty,
            columns,
          },
        },
        {
          name: "postgres-export",
          direction: "export",
          description: "export records to a postgres table",
          app: ["postgres"],
          options: [
            {
              key: "table",
              required: true,
              description: "the table to write records to",
            },
            // ...
          ],
          methods: {
            exportRecord,
            columns,
          },
        },
      ],
    });
  }
}

Actions

Actions are how you can add API endpoints to Grouparoo!

To learn more about actions, visit https://www.actionherojs.com/tutorials/actions

You likely want to authenticate your actions. You can use Grouparoo's middleware for this:

import { Action } from "actionhero";

export class AwesomeStatus extends Action {
  constructor() {
    super();
    this.name = "awesome:getStatus";
    this.description = "I let you know how awesome you are";
    this.inputs = {};
    this.middleware = ["authenticated-action"]; // <-- here

    // define the permissions needed to use this action
    this.permission = { topic: "record", mode: "read" };
  }

  async run() {
    return { awesome: true };
  }
}

If you want your action to require a logged-in web user or valid API Key, use the authenticated-action middleware. We will then check that this user/API Key has the proper permissions you specified.

Unlike actions directly provided by Grouparoo Core, Actions created by plugins will need to be manually added to the roues file. You do this in an Initializer within your Plugin:

import { Initializer, route } from "actionhero";

export class AwesomeInitializer extends Initializer {
  constructor() {
    super();
    this.name = "myPlugin:authentication";
  }

  async initialize() {
    route.registerRoute(
      "get", // The HTTP Verb
      "/v:apiVersion/awesome/status", // The Route (with variables)
      "awesome:getStatus" // the name of the action
    );
  }
}

Tasks

Actions are how you can add background jobs to Grouparoo!

To learn more about tasks, visit https://www.actionherojs.com/tutorials/tasks

Settings

You can define user-configurable settings for your plugins. These will be exposed to the Grouparoo Administrators in the "settings" menu. Settings are defined within an initializer.

// Define a setting for your Plugin
await api.plugins.registerSetting(pluginName, key, defaultValue, description);

// Read a setting
const { key, value, description, defaultValue } = await api.plugins.readSetting(
  pluginName,
  key
);

// Update a setting
await api.plugins.updateSetting(pluginName, key, value);

Settings are all of type string or number, so coerce them if needed. Setting keys and values are limited to 191 characters in length.

Iteration and Testing

When setting up Grouparoo, use pnpm install from the root of the monorepo to get everything installed. This used lerna to act like all the plugins are "regular" npm ones but there are really the local files.

When you add a new Plugin, it's version should be the same as all the other ones. This will allow lerna to pick up that local Plugin before it is published and allow you to iterate on the functionality. Run pnpm install after creating the new package and every time a dependency is added.

To make this new Plugin be used while running, add it to the package.json file of a Grouparoo App like the one in apps/staging-public. It needs to be added to both the dependencies and the plugins array at the bottom. After adding to these spots, run pnpm install from the root and npm run dev from the App.

It would be frustrating to have to run pnpm install every time there was a change. The running App, when run with npm run dev will automatically reboot itself with changes to the compiled Typescript. Run npm run watch (the same as node_modules/.bin/tsc --watch) from the Plugin directory to watch for changes to make this happen. If you do this, then file changes to the Plugin will show up momentarily in the running App. Sometimes, the App struggles to restart and you have to kill it and run npm run dev again.

When running tests for your new Plugin, use node_modules/.bin/jest __tests__/file/path.ts. This will automatically incorporate any changes to the library or test files in that Plugin.

Summary: Localhost

For localhost:3000 development (from monorepo root):

• pnpm install
• cd apps/staging-public && npm run dev
• cd plugins/@grouparoo/new-plugin && npm run watch

Now you can change files in plugins/@grouparoo/new-plugin and see them reflected on the server.

Summary: Testing

For jest testing (from monorepo root):

• cd plugins/@grouparoo/new-plugin
• node_modules/.bin/jest __tests__/file/path.ts

Change your files and tests and re-run command as needed.