Once upon a time — from the 1600s through the 1800s — Japan had a spy corps so famous we know their name today: the ninjas, intelligence agents serving the ruling Tokugawa family.
Over the last 75 years, however, as international spying and espionage has proliferated, Japan has mostly been on the sidelines of this global game. Defeat in World War II, and demilitarization afterward, meant that Japanese intelligence services were virtually nonexistent for decades.
Japan’s interest in spycraft has returned, however. In addition to a notable military expansion — as of last year, the country has aircraft carriers again — Japan is also ramping up its formal intelligence apparatus, as a response to what the country’s chief cabinet secretary has called “the drastically changing security environment” around it.
“Intelligence is a critical element of any national security strategy,” says MIT political scientist Richard Samuels, a leading expert on Japanese politics and foreign policy. “It’s just a question of how robust, and openly robust, any country is willing to make it.”
Examining the status of Japan’s intelligence efforts, then, helps us understand Japan’s larger strategic outlook and goals. And now Samuels has written a wide-ranging new history of Japan’s intelligence efforts, right up to the present. The book, “Special Duty: A History of the Japanese Intelligence Community,” is being published this week by Cornell University Press.
“Japan didn’t have a comprehensive intelligence capability, but they’re heading in that direction,” says Samuels, who is the director of the Center for International Studies and the Ford International Professor of Political Science at MIT. As firm as Japan’s taboo on military and intelligence activity once was, he adds, “that constraint is coming undone.”
Ruffians and freelance agents
Aside from the ninjas, who focused on domestic affairs, Japan’s international intelligence efforts have seen a few distinct phases: a patchy early period, a big buildup before World War II, the dismantling of the system under the postwar U.S. occupation, and — especially during the current decade — a restoration of intelligence capabilities.
Famously, Japan was closed off to much of the rest of the world until the late 19th century. It did not formally pursue international intelligence activities until the late 1860s. By the early 1900s, Japanese agents had found some success: They decoded Russian cables in the Russo-Japanese War of 1904-05 and cut off Russian raids during the conflict.
But as Samuels details in the book, during this period Japan heavily relied on a colorful array of spies and agents working on an unofficial basis, an arrangement that gave the country “plausible deniability” in case these operatives were caught.
“There was an interesting reliance upon scoundrels, ruffians, and freelance agents,” Samuels says.
Some of these figures were quite successful. One agent, Uchida Ryohei, founded an espionage group, the Amur River Society (also sometimes called the Black Dragon Society), which opened its own training school, created Japan’s best battlefield maps and conducted all manner of operations meant to limit Russian expansion. In the 1930s, another undercover agent, Doihara Kenji, became so successful at creating pro-Japanese local governments in northern China, that he became known as “Lawrence of Manchuria.”
Meanwhile, Japan’s official intelligence units had a chronic lack of coordination; they divided along military branches and between military and diplomatic bureaucracies. Still, in the decades before World War II, Japan leveraged some existing strengths in the study of foreign cultures — “The Japanese invented area studies before we did,” says Samuels — and used technological advances to make huge strides in information-gathering.
“They had strengths, they had weaknesses, they had official intelligence, they had nonofficial intelligence, but overall that was a period of great growth in their intelligence capability,” Samuels says. “That of course comes to a crashing halt at the end of the war, when the entire military apparatus was taken down. So there was this period immediately after the war where there was no formal intelligence.”
Japan’s subsequent postwar political reorientation toward the U.S. created many advantages for the country but was simultaneously a source of frustration to some. The country became an economic powerhouse while lacking the same covert capabilities as other countries.
“The Cold War was a period in which many Japanese in the intelligence world resented having to accommodate to American power in the intelligence world, and resented it,” Samuels says. “They had economic intelligence capability. They were very good at doing foreign economic analysis and were all over the world, but they were underperforming on the diplomatic and military fronts.”
The Asian pivot
In “Special Duty,” Samuels suggests three main reasons why any country reforms its intelligence services: Shifts in the strategic environment, technological innovations, and intelligence failures. The first of these seems principally responsible for the current revival of Japan’s intelligence operations.
As Samuels notes, some Japanese officials wanted to change the country’s intelligence structure during the 1980s — to little avail. The end of the Cold War, and the more complicated geopolitcal map that resulted, provided a more compelling rationale for doing so, without producing many tangible results.
Instead, more recent events in Asia have had a much bigger impact in Japan: namely, North Korean missile testing and China’s massive surge in economic and military power. In 2005, Samuels notes, Japan’s GDP was still twice that of China. A decade later, China’s economy was two and a half times as large as Japan’s, and its military budget was twice as big. U.S. power relative to China has also declined. Those developments have altered Japanese security priorities.
“There’s been a Japanese pivot in Asia,” Samuels notes. “That’s really very important.” Moreover, he adds, from the Japanese perspective, “The question about China is obvious. Is its rise going to be disjunctive, or is it going to be stabilizing?”
These regional changes have led Japan to chart a course of greater confidence in foreign policy — reflected in its growing intelligence function. Since 2013 in particular, after Prime Minister Shinzo Abe took office for a second time, Japan has built up its own intelligence function as never before, making operations more unified and better-supported. Japan still coordinates extensively with the U.S. in some areas of intelligence but is also taking intelligence matters into its own hands, in a way not seen for several decades.
As Samuels notes, Japan’s increasing foreign-policy independence is also supported by voters.
“Japanese public opinion has changed,” Samuels says. “They see the issues now, they talk about it now. Used to be, you couldn’t talk about intelligence in polite company. But people talk about it now, and they’re much more willing to go forward.”
“Special Duty” has been praised by other scholars in the field of Japanese security studies and foreign policy. Sheila Smith of the Council on Foreign Relations in Washington calls it a “truly wonderful book” that “offers much needed insight to academics and policymakers alike as they seek to understand the changes in Japan's security choices. ”
By looking at intelligence issues in this way, Samuels has also traced larger contours in Japanese history: first, an opening up to the world, then the alignment with the U.S. in the postwar world, and now a move toward greater capabilities. On the intelligence front, those capabilities include enhanced analysis and streamlined relations across units, heading toward the full spectrum of functions seen in the other major states.
“It’s been the assumption that the Japanese just don’t do [intelligence activities], except economics,” Samuels reflects. “Well, I hope after people see this book they will understand that’s no longer the case, and hasn’t been for some time.”
