Building a Custom Magik Completion Backend for Emacs

A custom Magik completion backend can make Emacs feel considerably more capable when working with GE Smallworld code. Instead of relying on a generic word list, completion can understand Magik identifiers, method names, class paths, local variables, collection types, and project-specific APIs. The result is quicker navigation through unfamiliar code and fewer interruptions while writing methods.

The most useful design combines Emacs’s completion framework with a lightweight Magik language index. That index may be generated from source files, exported from a running Smallworld session, or assembled from both sources. For Australian development teams working across Sydney, Melbourne, Brisbane, Perth, or remote sites, a local completion service can also reduce dependence on high-latency connections to an overseas development environment.

Why Magik Needs Language-Aware Completion

Basic Emacs completion can match words already present in the current buffer or open files. This is helpful for repetitive names, but it does not understand whether a symbol is a Magik method, a local variable, a slot, a global definition, or a class reference. A project containing several Smallworld products may also use naming conventions that are invisible to a generic completion engine.

Magik’s object-oriented model makes context especially important. A developer may want methods available on a particular class, slots inherited from a superclass, or procedures exposed by a framework package. A completion backend that treats every identifier as plain text will produce a long and noisy candidate list. A language-aware backend can rank local bindings first, then class methods, project symbols, framework definitions, and general Magik keywords.

Completion should also support the way Magik developers investigate existing systems. A candidate is more useful when it includes a short annotation, source location, declaring class, or method signature. Selecting a symbol could then lead directly to its definition through Emacs navigation commands, rather than requiring a separate search through large Smallworld code repositories.

Choosing the Emacs Completion Interface

The modern Emacs interface for this task is completion-at-point-functions, commonly abbreviated to CAPF. A CAPF function examines the current buffer position and returns a completion table, the beginning and end of the symbol, and optional metadata. It works with built-in completion commands and can also integrate with packages such as Corfu, Vertico, Orderless, and Company through the appropriate adapters.

A minimal CAPF has the following shape:

(defun magik-completion-at-point ()
  (when (derived-mode-p 'magik-mode)
    (let* ((bounds (bounds-of-thing-at-point 'symbol))
           (start (car bounds))
           (end   (cdr bounds)))
      (when bounds
        (list start end
              (magik-completion-table
               (buffer-substring-no-properties start end))
              :exclusive 'no
              :annotation-function #'magik-annotation)))))

The :exclusive value of no allows other completion functions to contribute candidates if the Magik backend has no useful result. This is valuable in mixed files or older buffers where the major mode cannot confidently identify the current context. Register the function locally rather than globally so that ordinary Emacs buffers are unaffected:

(add-hook 'magik-mode-hook
          (lambda ()
            (add-hook 'completion-at-point-functions
                      #'magik-completion-at-point nil t)))

A backend can return a static list, a function-based completion table, or a dynamic table backed by an index. Function-based tables are generally preferable for a large codebase because they can filter candidates incrementally and avoid copying thousands of symbols into every buffer.

Building A Useful Magik Symbol Index

The completion table needs reliable data. A practical first version can scan Magik source files and record symbol names, defining files, line numbers, classes, methods, and categories. A parser does not have to understand every part of the language immediately. It can begin with method declarations, class declarations, global names, common keywords, and simple slot definitions, then become more precise as real completion failures are discovered.

Regular expressions may be sufficient for an initial prototype, provided they account for comments, quoted strings, method modifiers, and multiline declarations. A more durable solution tokenises the source before extracting definitions. This avoids indexing a method name that appears inside a comment or a documentation example. The index should retain the original spelling because Magik codebases may use conventions that differ from Emacs’s usual case-folding behaviour.

A record might contain fields like these:

(:name " Smallworld_name"
 :kind method
 :class "gis::some_class"
 :file "/project/src/example.magik"
 :line 184
 :signature "_method object.some_method(arg1, arg2)")

The leading space in this example is illustrative only; the backend should normalise names consistently while preserving the spelling displayed to the programmer. Candidate metadata can expose kind and class, while an annotation-function adds compact labels such as [method], [slot], or [keyword].

For larger repositories, create the index in a background process or during project startup. A command such as magik-refresh-index can rescan changed files and write a cache under the project directory. Avoid storing proprietary source content in a shared or cloud-based cache. Australian organisations handling customer, utility, or government data should consider the Privacy Act 1988 and the Australian Privacy Principles when deciding what telemetry or source information leaves the workstation.

Adding Context From The Current Buffer

A completion backend becomes much more useful when it recognises the text immediately before point. At minimum, distinguish ordinary symbol completion from member completion. If the source contains an object expression followed by a method separator, the backend should query methods associated with that expression rather than return every symbol in the project.

The exact separator and syntax should be confirmed against the Magik dialect and coding standards used by the target Smallworld installation. A context analyser can return a structure such as:

(:kind member
 :receiver "record"
 :prefix "get_")

For a local variable, the analyser can search the current method for declarations and assignments. For a known class or constructor, it can consult the inheritance graph. When the receiver cannot be resolved statically, the backend can return a broader set of methods while marking the result as approximate. This graceful fallback is preferable to suppressing completion altogether in a dynamically typed environment.

Magik developers often work with collections and database-backed objects, so completion should recognise common selectors, iterators, query helpers, and domain-specific methods. A project may have generated classes that are absent from hand-written source files. Importing runtime metadata from a controlled Smallworld session can fill this gap, but the backend should label runtime-derived results clearly and cope with an unavailable session.

Local variables deserve high priority because they are frequently the intended completion target. A sensible ranking order is current-method locals, parameters, receiver methods, imported project symbols, framework APIs, and language keywords. Fuzzy matching can be offered through Orderless, while the backend remains responsible for producing accurate candidates and useful annotations.

Connecting Emacs To A Runtime Or Language Service

There are two broad ways to obtain Magik information. The simplest is an offline index generated from source. The richer approach communicates with a running Magik environment or a dedicated language service. A runtime connection can answer questions about actual objects, loaded packages, inheritance, and methods that are generated or installed during application startup.

Keep runtime requests asynchronous where possible. Completion must feel immediate, particularly when a developer is working over a corporate VPN or a connection between an Australian office and a hosted environment in another region. A synchronous request that waits several seconds can make ordinary typing unpleasant. Use a short timeout, cache recent responses, and fall back to local symbols when the runtime cannot answer.

An Emacs process filter or JSON-based protocol can provide a clean boundary:

(defun magik-runtime-completions (context callback)
  ;; Send CONTEXT to the Magik service.
  ;; Invoke CALLBACK with a list of completion records.
  (magik-send-request
   "complete"
   context
   (lambda (response)
     (funcall callback (magik-decode-candidates response)))))

The transport does not need to be complicated for a first release. A local TCP process, standard input/output subprocess, or an existing Smallworld integration may be enough. Define explicit request fields for project, buffer context, receiver, prefix, and client version. Versioning matters because Smallworld installations can differ substantially between customer environments and supported releases.

In the Australian market, geospatial systems may support mining operations in Western Australia, council assets in New South Wales, or transport infrastructure in Victoria. Those deployments can have different network controls and operational constraints. Design the backend so that indexing and basic completion continue to work offline, while runtime inspection remains an optional enhancement approved by the relevant organisation.

Making Candidate Selection And Navigation Practical

A completion list should help a developer make a decision quickly. An annotation function can show the candidate kind, declaring class, or package without making the displayed name unwieldy. A documentation function can show the method signature and a short source comment in the minibuffer or a side window. These features turn completion into a lightweight discovery tool for a large Magik application.

Candidate categories should be represented with metadata rather than encoded into the visible symbol name. For example:

(defun magik-annotation (candidate)
  (let ((kind (get-text-property 0 'magik-kind candidate))
        (owner (get-text-property 0 'magik-owner candidate)))
    (format " %-10s %s" (or kind "") (or owner ""))))

The exact storage method can vary. A completion table may return propertised strings, use an internal hash table keyed by candidate name, or provide metadata through a custom table function. Test the implementation with duplicate names, candidates differing only by case, and symbols inherited from several classes.

Completion should connect naturally with definition lookup. Add commands such as magik-find-definition and magik-describe-symbol, then bind them alongside the completion command in magik-mode. A selected candidate can carry its file and line location, allowing xref integration or a direct jump. This is particularly valuable when a Melbourne-based support team is maintaining a system originally implemented by a Perth or Sydney project group.

Respect Australian spelling and team conventions in documentation, diagnostics, and configuration examples. The code itself must follow the conventions required by the Magik project, but messages such as “initialise index” and “optimise cache” can match the language used by local engineering teams. Small details like these make an internal tool feel properly maintained rather than imported without adaptation.

Testing, Performance, And Safe Deployment

Test the backend with representative Magik files rather than a few artificial examples. Include nested methods, comments containing code-like text, quoted strings, long identifiers, incomplete expressions, inherited methods, generated classes, and files with unusual encoding. Verify that completion works at the beginning of a symbol, in the middle of a name, after a member separator, and when no candidate is available.

Measure latency from keystroke to candidate display. Local index lookups should generally be fast enough for interactive use, while remote or runtime completion should be cached and cancellable. Do not refresh the entire repository on every request. Track file modification times, invalidate only affected records, and rebuild inheritance relationships when class definitions change.

Security deserves explicit treatment in enterprise environments. Do not send complete buffers to a remote service when a prefix and small context window are sufficient. Provide a setting that disables runtime queries, document where caches are stored, and ensure logs do not retain sensitive source or database values. Teams working with public-sector or critical-infrastructure data may require review under internal security policies in addition to obligations under Australian privacy legislation.

Package the backend as a small Emacs library with customisable variables for the index path, runtime endpoint, timeout, case sensitivity, and cache policy. Include an example configuration and automated tests using ERT. A useful first release can provide source-index completion, annotations, and definition lookup; runtime object inspection and advanced type inference can be added after the core workflow is stable.

Install the backend in a disposable Magik project first, then measure its usefulness against real editing tasks: locating unfamiliar methods, completing framework APIs, and moving from a candidate to its definition. HydePark Consulting can help teams assess their Emacs and Magik workflow, adapt indexing to an existing Smallworld codebase, and integrate the extension with broader development practices. Start with a focused local index, keep runtime access optional, and let reliable completion become the foundation for deeper Magik tooling.