One of the common arguments I hear about avoiding XA distributed transactions is due to their complexity. In this series of blog posts I’ll examine that claim by looking at three versions of the same application. The first version of the application ignores data consistency issues and operates as though failures essentially don’t occur. This unfortunately is a pretty common practice due to the perceived complexity of introducing distributed transactions into an application. The second version adopts the saga pattern that is all the rage. It uses Eclipse MicroProfile Long Running Actions to implement the saga. Unlike the toy examples used to illustrate how sagas work, this version will include the necessary logic to actually be able to complete or compensate a transaction. Finally, the third version will use the XA pattern to ensure data consistency.
Basic Problem to Solve
The application I’ll use to illustrate the issues associated with ensuring data consistency is one that provides the transfer of money from one account to another account where each account is serviced by a different microservice. Fundamentally a very simple application if you ignore failures. The flow is basically a Transfer microservice:
1. Accepts a request to transfer an amount of money from one account to another account
2. Makes a request to withdraw the amount from the first account
3. Makes a request to deposit the amount in the second account
4. Returns success
A very simple application, what could possibly go wrong?
Simplistic Application Without Considering Failures
Let’s look first at a possible simple Transfer microservice. It offers a single service named “transfer”, which transfers money from an account in one microservice (department1) to an account in a different microservice (department2). Here is a simple Spring Boot based teller service that handles transferring the money:
@RestController
@RequestMapping("/transfers")
@RequestScope
public class TransferResource {
private static final Logger LOG = LoggerFactory.getLogger(TransferResource.class);
@Autowired
RestTemplate restTemplate;
@Value("${departmentOneEndpoint}")
String departmentOneEndpoint;
@Value("${departmentTwoEndpoint}")
String departmentTwoEndpoint;
@RequestMapping(value = "transfer", method = RequestMethod.POST)
public ResponseEntity<?> transfer(@RequestBody Transfer transferDetails) throws TransferFailedException {
ResponseEntity<String> withdrawResponse = null;
ResponseEntity<String> depositResponse = null;
LOG.info("Transfer initiated: {}", transferDetails);
try {
withdrawResponse = withdraw(transferDetails.getFrom(), transferDetails.getAmount());
if (!withdrawResponse.getStatusCode().is2xxSuccessful()) {
LOG.error("Withdraw failed: {} Reason: {}", transferDetails, withdrawResponse.getBody());
throw new TransferFailedException(String.format("Withdraw failed: %s Reason: %s", transferDetails, withdrawResponse.getBody()));
}
} catch (Exception e) {
LOG.error("Transfer failed as withdraw failed with exception {}", e.getLocalizedMessage());
throw new TransferFailedException(String.format("Withdraw failed: %s Reason: %s", transferDetails, Objects.nonNull(withdrawResponse) ? withdrawResponse.getBody() : withdrawResponse));
}
try {
depositResponse = deposit(transferDetails.getTo(), transferDetails.getAmount());
if (!depositResponse.getStatusCode().is2xxSuccessful()) {
LOG.error("Deposit failed: {} Reason: {} ", transferDetails, depositResponse.getBody());
LOG.error("Reverting withdrawn amount from account {}, as deposit failed.", transferDetails.getFrom());
redepositWithdrawnAmount(transferDetails.getFrom(), transferDetails.getAmount());
throw new TransferFailedException(String.format("Deposit failed: %s Reason: %s ", transferDetails, depositResponse.getBody()));
}
} catch (Exception e) {
LOG.error("Transfer failed as deposit failed with exception {}", e.getLocalizedMessage());
LOG.error("Reverting withdrawn amount from account {}, as deposit failed.", transferDetails.getFrom());
redepositWithdrawnAmount(transferDetails.getFrom(), transferDetails.getAmount());
throw new TransferFailedException(String.format("Deposit failed: %s Reason: %s ", transferDetails, Objects.nonNull(depositResponse) ? depositResponse.getBody() : depositResponse));
}
LOG.info("Transfer successful: {}", transferDetails);
return ResponseEntity
.ok(new TransferResponse("Transfer completed successfully"));
}
/**
* Send an HTTP request to the service to withdraw amount from the provided account identity
*
* @param accountId The account Identity
* @param amount The amount to be withdrawn
*/
private void redepositWithdrawnAmount(String accountId, double amount) {
URI departmentUri = UriComponentsBuilder.fromUri(URI.create(departmentOneEndpoint))
.path("/accounts")
.path("/" + accountId)
.path("/deposit")
.queryParam("amount", amount)
.build()
.toUri();
ResponseEntity<String> responseEntity = restTemplate.postForEntity(departmentUri, null, String.class);
LOG.info("Re-Deposit Response: \n" + responseEntity.getBody());
}
/**
* Send an HTTP request to the service to withdraw amount from the provided account identity
*
* @param accountId The account Identity
* @param amount The amount to be withdrawn
* @return HTTP Response from the service
*/
private ResponseEntity<String> withdraw(String accountId, double amount) {
URI departmentUri = UriComponentsBuilder.fromUri(URI.create(departmentOneEndpoint))
.path("/accounts")
.path("/" + accountId)
.path("/withdraw")
.queryParam("amount", amount)
.build()
.toUri();
ResponseEntity<String> responseEntity = restTemplate.postForEntity(departmentUri, null, String.class);
LOG.info("Withdraw Response: \n" + responseEntity.getBody());
return responseEntity;
}
/**
* Send an HTTP request to the service to deposit amount into the provided account identity
*
* @param accountId The account Identity
* @param amount The amount to be deposited
* @return HTTP Response from the service
*/
private ResponseEntity<String> deposit(String accountId, double amount) {
URI departmentUri = UriComponentsBuilder.fromUri(URI.create(departmentTwoEndpoint))
.path("/accounts")
.path("/" + accountId)
.path("/deposit")
.queryParam("amount", amount)
.build()
.toUri();
ResponseEntity<String> responseEntity = restTemplate.postForEntity(departmentUri, null, String.class);
LOG.info("Deposit Response: \n" + responseEntity.getBody());
return responseEntity;
}
}
Note that this simplistic implementation of the transfer service at least considers the possibility that the deposit service fails and if so attempts to redeposit the withdrawn amount back into the source account. However as should be obvious, it’s possible that the redeposit fails thus leaving the funds in limbo.
Here is the code providing the REST service interface for withdraw for Department1:
@RequestMapping(value = "/{accountId}/withdraw", method = RequestMethod.POST)
public ResponseEntity<?> withdraw(@PathVariable("accountId") String accountId, @RequestParam("amount") double amount) {
try {
this.accountOperationService.withdraw(accountId, amount);
return ResponseEntity.ok("Amount withdrawn from the account");
} catch (NotFoundException e) {
return ResponseEntity.status(HttpStatus.NOT_FOUND).body(e.getMessage());
} catch (UnprocessableEntityException e) {
LOG.error(e.getLocalizedMessage());
return ResponseEntity.status(HttpStatus.UNPROCESSABLE_ENTITY).body(e.getMessage());
} catch (Exception e) {
LOG.error(e.getLocalizedMessage());
return ResponseEntity.internalServerError().body(e.getLocalizedMessage());
}
}
The withdraw service calls the withdraw method on the accountOperationService, Here is the code providing the withdraw method used by the above REST service. It uses an injected EntityManager for JPA:
/**
* Service that connects to the accounts database and provides methods to interact with the account
*/
@Component
@RequestScope
@Transactional
public class AccountOperationService implements IAccountOperationService {
private static final Logger LOG = LoggerFactory.getLogger(AccountOperationService.class);
@Autowired
EntityManager entityManager;
@Autowired
IAccountQueryService accountQueryService;
@Override
public void withdraw(String accountId, double amount) throws UnprocessableEntityException, NotFoundException {
Account account = accountQueryService.getAccountDetails(accountId);
if (account.getAmount() < amount) {
throw new UnprocessableEntityException("Insufficient balance in the account");
}
LOG.info("Current Balance: " + account.getAmount());
account.setAmount(account.getAmount() - amount);
account = entityManager.merge(account);
entityManager.flush();
LOG.info("New Balance: " + account.getAmount());
LOG.info(amount + " withdrawn from account: " + accountId);
}
@Override
public void deposit(String accountId, double amount) throws NotFoundException {
Account account = accountQueryService.getAccountDetails(accountId);
LOG.info("Current Balance: " + account.getAmount());
account.setAmount(account.getAmount() + amount);
account = entityManager.merge(account);
entityManager.flush();
LOG.info("New Balance: " + account.getAmount());
LOG.info(amount + " deposited to account: " + accountId);
}
}
The withdraw method gets the current account balance and checks for sufficient funds and throws an exception if insufficient funds. Otherwise it updates the account balance and saves the account. We can also see the deposit method which gets the current account balance, adds the amount to deposit and saves the updated account information.
What About Failures?
The developer of this teller service realizes there might be some failure scenarios to handle. For example, what happens if the deposit request fails? The developer solved that by having the teller service takes the corrective measure of redepositing the money back into the first account. Problem solved. But what happens if between the time of the withdrawal request and the request to redeposit the funds the teller microservice dies? What happens to the funds that were withdrawn? As it stands, they’re lost!
The developer could solve this problem by creating a table of pending operations that could be examined when the teller microservice starts up. But that would also mean that the deposit service must be idempotent as the only thing the teller service can do is retry the deposit request until it succeeds at which point it would remove the entry from its pending operations table. Until the deposit succeeds, the funds are basically in limbo and inaccessible to the account owner or anyone else.
So far, the developer has only handled some of the possible failures by adding error recovery logic into their microservice. And this is only for a trivial microservice. As more state information is updated, more complex recovery mechanisms may need to be added to the microservice. In the next post, we’ll look at how we can apply the saga pattern to solve this data consistency problem using Eclipse MicroProfile Long Running Actions, coordinated by MicroTx.
Source: oracle.com
Posts
