A customer who enters the system at time will still be in the system at time with probability . Let be a Poisson process with intensity and , . Then , and so for each nonnegative integer

In the case where , that is, the service times are exponentially distributed with rate , we have

and hence

The expected number of customers at time is then

Moreover, we observe that

which is the same as the stationary distribution of derived from the generalized global balance equations

(normalized by the condition ).

]]>where . Then is irreducible and aperiodic, so there exists a unique stationary distribution satisfying

where denotes the row of . From the global balance equation we see that , and normalization yields

To actually compute , we could write where the columns of are linearly independent eigenvectors of and is a diagonal matrix whose entries are the corresponding eigenvalues. This is possible because the Perron-Frobenius theorem implies that the eigenspace corresponding to the eigenvalue is one-dimensional, and the rank of is two. Instead, we will compute the generating function of . Let

then

where

Partial fraction decomposition yields

and so we have

Similar computations yield

and it follows that

]]>Suppose customers arrive to a queue according to a Bernoulli process; that is, at each time , a customer arrives with probability , independent of past or future arrivals. The probability that a customer finishes service at time , conditioned on that customer having been in service at time , is . Let be the arrival process and the departure process. The interarrival times are , with . Conditioned on , we have , and so by independence of arrivals, the are i.i.d. random variables. A similar argument shows that the service times are i.i.d. random variables.

Let be the number of customers in the system at time , after arrivals and before departures. It follows from the memoryless property of the geometric distribution that is a Markov chain; that is, for any nonnegative integers , , and and any , we have

At each time , at most one customer can arrive and at most one customer can depart. It follows that is a birth-death chain, with transition probabilities given by

Suppose is an invariant measure for the transition matrix , that is . Assume WLOG that , then and

By inspection, this recurrence has solution

Now, is positive recurrent iff , which is true precisely when

The offered load to the system is then

Assuming , let and define . By construction, and , so that is the (unique) stationary distribution of . A straightforward computation yields

The limiting mean number of customers in the system is given by

Invoking Little’s Law, we compute the mean sojourn time of customers by

]]>

– that is, the slope of the line segment joining and differs from the slope of the line segment joining and . Define such that and the graph of is

That is, is a piecewise linear function connecting the points – like the functions used in the previous post. Then is differentiable on . Moreover, if is a strictly increasing sequence with satisfying the aforementioned slope assumption, then defining as above, we see that is continuous on and differentiable at all but countably many points.

The paths of Brownian motion are even more pathological – they are with probability differentiable nowhere! Let be a standard Brownian motion and

We will show that . For each integer and , set

and

Suppose , then since

exists, we may choose , such that

Let be a positive integer with , and put

Then

so that

and similarly

It follows that

Now, the increments are i.i.d. with distribution, so

from which we have

It follows that

Let , then so . Fatou’s lemma then yields

from which we conclude.

]]>- almost surely.
- has independent increments, i.e. for , the random variables
are independent.

- For ,
- The map is almost surely continuous.

In this post I follow the method in Sidney Resnick’s *Adventures in Stochastic Processes* to construct Brownian motion on using i.i.d. standard normal random variables. The key to this construction is the following lemma:

Lemma 1Suppose and are random variables such thatThen there exists a random variable such that

*Proof:* Define and suppose is independent of . Define by

so that

Then as , we have

so that and are uncorrelated and hence independent (as they are normally distributed).

Now let

be independent random variables with

Let , , and define using so that and are i.i.d. random variables. Suppose

are defined such that

For each , define such that

and the sequence has independent increments. For each and define the processes

and

That is,

and is linear on each interval . Let be the maximum deviation of and on , partitioned by the intervals , that is,

For each we have

where , and so

For we compute

where and is the probability density of . Let , then we have

Since

from the first Borel-Cantelli lemma we have

from which

This implies that the sequence of functions on is Cauchy, as for

Since is complete, we conclude that exists, with probability , and by construction is Brownian motion on .

]]>Let be the number of customers in the system at time , then is a Markov chain with transition probabilities

Suppose is an invariant measure for , that is, . A stationary distribution exists for iff , in which case . Assume WLOG that , then the global balance equations are

It follows that

Let be the generating function of , then multiplying the recurrence by and summing over yields

so that

Solving for , we have

from which it follows that

We compute

and hence

]]>If then

so is a bounded operator. It follows that there exists a unique adjoint operator satisfying for all . Since has finite Lebesgue measure, , and thus

It follows that and hence by Fubini’s theorem,

from which we see that .

If with , then for any , Hölder’s inequality yields

so that is Hölder continuous. Moreover, the kernel of is and

so . It follows that is a Hilbert-Schmidt operator and therefore is compact. It follows then from the Fredholm alternative (see \texttt{http://bit.ly/1Uh8BgP}) that if , then is an eigenvalue of . That is, for all . If , then as we must have . By the fundamental theorem of calculus, , and so . Differentiating both sides of the eigenvalue equation yields , and hence for some . Since , we have , and therefore . It follows that the spectral radius of , , is zero.

If is a bounded operator on a Hilbert space and then

so

and hence . By symmetry, , whence . It follows that . Recall Gelfand’s formula for the spectral radius of a bounded operator :

If is a self-adjoint operator and then

which implies . By induction it follows that for all and hence

If is normal, then by induction , and as is self-adjoint, , from which we see that the spectral radius of a normal operator is equal to its operator norm.

Since the spectral radius of the Volterra operator is zero and its operator norm is positive, we conclude that is not a normal operator.

]]>

* Customers of two types arrive to a system according to independent Poisson processes with rates and , respectively. The system has two servers, each serving only one customer type with rates for customer type and for customer type . There is room for only one customer to wait in a queue which is shared by the two servers. When the queue position is occupied, arriving customers are lost.*

Let be a CTMC describing this process

- Define the state space.
- Write the balance equations.
- What is the long-run utilization of server ?

Let the state space be where the first symbol in each state denotes the buffer status, the second symbol denotes the status of server , and the third symbol the status of server . The generator matrix of transition rates is given by

The global balance equations are given by

The utilization of server is given by

Consider a single server queue where customers arrive according to a Poisson process with intensity and request i.i.d. service times. The server is subject to failures and repairs. The lifetime of a working server is an random variable, while the repair time is an . random variable. Successive lifetimes and repair times are independent, and are independent of the number of customers in the queue. When the server fails, all the customers in the queue are forced to leave, and while the server is under repair no new customers are allowed to join. Model this as a CTMC.

- What is the state space?
- What are the balance equations? Show how to solve them.
- What is the long run fraction of time the server is idle but not operational?
- What fraction of incoming customers leave without service?

Let be a CTMC on state space with transition rates

The balance equations are

Since , we have and hence . It follows that

Consider an M/M/1/2 queue with impatient customers. As soon as customer ’s sojourn time in the system exceeds , he/she leaves the system without receiving service. (Customers can leave the system even when they are in service.) Suppose that are i.i.d. random variables exponentially distributed with rate . Customers arrive with rate and the service rate is .

- Model this system as a CTMC and give the steady-state equations.
- Give an expression for the long-run average fraction of customers who are not admitted to the queue or choose to leave with either no service or incomplete service after they join.

Let be a CTMC on state space . Transitions to states and occur when a customer abandons. The transition rates are

The transition matrix of the embedded Markov chain is

Solving and yields

The long run fraction of customers who are not admitted is and the effective arrival rate is

The long run fraction of customers who abandon is

Adding this two quantities yields

the long run fraction of throughput lost.

Customers arrive at a service station according to a Poisson process with rate . Servers arrive at this station according to an independent renewal process with i.i.d. interarrival times with mean and second moment . Each incoming server removes each of the waiting customers with probability in an independent fashion, and departs immediately. Let be the number of customers at the service station after the server departs, and let be the number of customers at the station at time .

- Show that is a DTMC.
- Compute the limiting value of as .
- Show that is a Markov regenerative process.
- Compute the limiting value of as .

]]>

Theorem 1Let be an open connected set and an integral operator with kernel , that is,where

Then is compact.

*Proof:* Let be an orthonormal basis of , then write the kernel as

where

Define the sequence of integral operators on by

where

It is clear that , so the are of finite rank and hence compact. For , we have

As , it follows that

We conclude that is compact as the limit of a sequence of compact operators in the operator norm topology.

]]>
Now, the more interesting ones. A discrete-time process is said to be a **martingale** with respect to a filtration if

- is adapted to .
- is integrable for all (i.e. ).
- for all .

Recall that the conditional expectation of a random variable with respect to a -algebra , denoted , is any random variable that satisfies for all , that is,

For example, if is an i.i.d. sequence with and , then

so the unbiased 1- random walk is a martingale. For a biased random walk on with , it is clear that is not a martingale. However, letting where , we have

and

so that is integrable, and

It follows that is a martingale. If and then

so is a Markov chain. Further, for any ,

so that is a harmonic function. In general, if is a harmonic function for a Markov chain , then is a martingale. This follows from the Markov property:

]]>