...
Topics are created in TMS using both a namespace and a topic name. Note that while it is possible to fetch from from the system namespace, attempting to publish to the system namespace will result in an error.
Currently, as Because TMS does not use authorization, you can only create topics in the same namespace as that your application is running in. For that reason, you do not require the name of the current namespace is not required to create a topic. However, you can publish to all topics in all namespaces (except the system namespace, as noted above) and fetch from all topics in all namespaces.
...
Non-transactional messaging in TMS works in a manner very similar to other messaging systems:
publishPublish a message (also called a payload) to a topic.
fetchFetch (also called subscribe or consume) from a topic.
TMS provides strong ordering guarantees for the consumers of a topic:
...
With transactional messages, messages are not available (published) until the transaction has been successfully committed.
For example, a data pipeline might follow these steps:
Open a transaction
Do some work
Publish a message
Do additional work
Commit the transaction
If the transaction is successful: , the message is now visible to transactional consumers.
If the transaction is unsuccessful: , the message is rolled back and is never seen by transactional consumers.
However, as noted above, non-transactional consumers see all messages of a topic, including the messages that were published and are currently in a transaction.
...
If it were to publish to a topic non-transactionally, a problem can arise as there is no guarantee that the notification will be published only after the dataset commit. If it were to publish transactionally to a TMS topic, there is the guarantee that transaction consumers will only see the notification if the write to the dataset is successfully committed:
...
Transactional Example
Currently, TMS : only supports explicit transactions
...
and
...
does not support publishing from a long-running transaction, such as a mapper, reducer, or Spark executor.
Java API
Javadocs describing the TMS Java API are available in the package io.cdap.cdap.api.messaging:
MessagingAdmin: Provides topic administration functions
MessagingContext: Provides access to the Transactional Messaging System
MessagePublisher: Provides message publishing functions
MessageFetcher: Provides message fetching functions
Limitations
Currently, TMS does not use authorization , and does not allow creating topics outside of the current namespace.