Kiyoshi Ito (of Ito calculus fame) is dead.
(When? I'm not sure. The New York Times says Monday, the 17th, but the Japan Times published an obituary on Saturday the 15th which said he died "Monday" -- so I'm guessing the 10th.)
See the obituaries, the MacTutor biography and Wikipedia article, or this Notices article upon his receipt of the Gauss Prize for an idea of his contributions.
Showing posts with label stochastic processes. Show all posts
Showing posts with label stochastic processes. Show all posts
23 November 2008
28 May 2008
Stochastic generation of ideas
I'm reading about continuous time Markov processes, from Adventures in Stochastic Processes: The Random World of Happy Harry by Sidney Resnick. This book has some of the most amusing problems I've come across in a while; many of the problems involve a character known as "Happy Harry", who owns a greasy spoon near some university that occasionally reveals itself to be Cornell. This post does not involve one of those problems.
A discrete-time Markov chain is what people usually mean when they say "Markov chain". This is a sequence of elements X0, X1, X2, ... selected from some state space S = (s1, s2, ...), where
P(Xn = sj | Xn-1 = si) = pij
(the pij are called transition probabilities) and Xn does not depend on the history before Xn-1. Now, from any discrete-time Markov chain it's possible to construct a continuous-time Markov chain. We simply assign a parameter λ(j) to each state sj. When the discrete-time Markov chain finds itself in state j, we wait for a time which is exponentially distributed with parameter λ(j) (and therefore mean 1/λ(j)); after this time passes we proceed into another state according to the transition probabilities.
There's a problem, though, as Resnick explains in Section 5.2 -- it can happen that the chain "blows up", i. e. it makes infinitely many transitions in finite time.
The "pure birth" process has state space {1, 2, 3, ...}; the transition probabilities are pn,n+1 = 1 with all others zero; the parameters are λ(n) = λn. That is, the process waits in state n for a time which is exponentially distributed with mean 1/(λn) and then advances to state n+1. Not surprisingly, the population grows exponentially fast -- if the population is at n currently, it grows at rate λn.
But if you let λ(n) = λn2, then the process "blows up" almost surely. This isn't surprising from the point of view of a differential equation -- if the population is n, it grows at rate λn2. If we ignore the stochastic elements and model this as an ordinary differential equation, the population P satisfies dP/dt = λP2, and solutions to this have vertical asymptotes. Something similar (I don't pretend to know all the details) happens in the stochastic case.
I then thought, when would you get a population that grows like this? Consider not a population of living beings, but a population of ideas. And let us assume that new ideas come into being when two old ideas combine. Then in some idealized world where all pairs of ideas are constantly interacting, one might expect that the rate at which new ideas exist is proportional to the number of pairs of ideas. Of course, this is a silly model, because ideas don't interact all by themselves, but rather in people's brains -- and the population of people grows just exponentially. Still, this feels like it could explain why certain ideas seem to "come out of nowhere" -- at first in some area of intellectual inquiry there isn't much there because new ideas require new insights, but at some point combining old insights in new ways becomes a viable way to come up with new ideas.
(Of course, checking this against reality would be quite difficult. For one thing, how do you count ideas?)
A discrete-time Markov chain is what people usually mean when they say "Markov chain". This is a sequence of elements X0, X1, X2, ... selected from some state space S = (s1, s2, ...), where
P(Xn = sj | Xn-1 = si) = pij
(the pij are called transition probabilities) and Xn does not depend on the history before Xn-1. Now, from any discrete-time Markov chain it's possible to construct a continuous-time Markov chain. We simply assign a parameter λ(j) to each state sj. When the discrete-time Markov chain finds itself in state j, we wait for a time which is exponentially distributed with parameter λ(j) (and therefore mean 1/λ(j)); after this time passes we proceed into another state according to the transition probabilities.
There's a problem, though, as Resnick explains in Section 5.2 -- it can happen that the chain "blows up", i. e. it makes infinitely many transitions in finite time.
The "pure birth" process has state space {1, 2, 3, ...}; the transition probabilities are pn,n+1 = 1 with all others zero; the parameters are λ(n) = λn. That is, the process waits in state n for a time which is exponentially distributed with mean 1/(λn) and then advances to state n+1. Not surprisingly, the population grows exponentially fast -- if the population is at n currently, it grows at rate λn.
But if you let λ(n) = λn2, then the process "blows up" almost surely. This isn't surprising from the point of view of a differential equation -- if the population is n, it grows at rate λn2. If we ignore the stochastic elements and model this as an ordinary differential equation, the population P satisfies dP/dt = λP2, and solutions to this have vertical asymptotes. Something similar (I don't pretend to know all the details) happens in the stochastic case.
I then thought, when would you get a population that grows like this? Consider not a population of living beings, but a population of ideas. And let us assume that new ideas come into being when two old ideas combine. Then in some idealized world where all pairs of ideas are constantly interacting, one might expect that the rate at which new ideas exist is proportional to the number of pairs of ideas. Of course, this is a silly model, because ideas don't interact all by themselves, but rather in people's brains -- and the population of people grows just exponentially. Still, this feels like it could explain why certain ideas seem to "come out of nowhere" -- at first in some area of intellectual inquiry there isn't much there because new ideas require new insights, but at some point combining old insights in new ways becomes a viable way to come up with new ideas.
(Of course, checking this against reality would be quite difficult. For one thing, how do you count ideas?)
21 February 2008
Politics and winding numbers
Confusion about the changing positions of political parties in the U.S., from Andrew at Statistical Modeling, Causal Inference, and Social Science.
There are fairly standard models that allow one to plot political opinions in a two-dimensional "issue space", with the two dimensions being roughly "social" and "economic"; the two-party system dictates that these essentially get projected down to a single dimension. The post alludes to them, and refers to an argument that the Democrats and Republicans may have switched positions in the last century or so via a rotation of 180 degrees in this space. (As counterintuitive as it seems given the current ideological stances of the major parties, the Republicans started out as the anti-slavery party.) Andrew is not convinced -- the argument seems to rely on an assumption that states remain constant in their political leanings, which isn't true -- but it at least seems like something that could happen.
So in another century, could the parties rotate all the way around and get back where they started? And if the alignment of the two major parties in issue space in 2100 ends up being the same as that in 1900, is it more likely to happen by the rotation continuing in the direction it's going, or by reversal? This is basically a question about winding numbers, in a sense I really don't want to make precise because it's kind of silly.
There are fairly standard models that allow one to plot political opinions in a two-dimensional "issue space", with the two dimensions being roughly "social" and "economic"; the two-party system dictates that these essentially get projected down to a single dimension. The post alludes to them, and refers to an argument that the Democrats and Republicans may have switched positions in the last century or so via a rotation of 180 degrees in this space. (As counterintuitive as it seems given the current ideological stances of the major parties, the Republicans started out as the anti-slavery party.) Andrew is not convinced -- the argument seems to rely on an assumption that states remain constant in their political leanings, which isn't true -- but it at least seems like something that could happen.
So in another century, could the parties rotate all the way around and get back where they started? And if the alignment of the two major parties in issue space in 2100 ends up being the same as that in 1900, is it more likely to happen by the rotation continuing in the direction it's going, or by reversal? This is basically a question about winding numbers, in a sense I really don't want to make precise because it's kind of silly.
Subscribe to:
Posts (Atom)
