Question:
There is an object that inherits from IEnumerable
. In MoveNext
method of this class, a StackOverflowException
occurs on the line with Regex
. The regular expression itself looks for text matches in files. Remarkably, everything works on Windows 10 x64 with .Net 4.6, the error occurs on Windows 7 x64 with .Net 4.0.
I have 2 guesses why this is happening:
-
First: perhaps due to differences in the implementation of the
Regex
method in .Net 4.0 and .Net 4.6,Regex
takes more memory on the stack and therefore crashes with an exception. -
Second: maybe the stack size in Win10 is different from that in Win7.
How to check current stack size? How to check available stack size? And what else could be the reason for this error?
The number of files in the IEnumerable
to be searched is 2760 ( _objEnumerator.Count
), each file is loaded in advance and stored as a string in this very IEnumerable
. Below is an example code:
private class MyEnumerator : IEnumerable
{
public bool MoveNext()
{
if (_objEnumerator == null)
{
_objEnumerator = _objects.GetEnumerator();
}
Match m;
if (_current == null)
{
if (!_objEnumerator.MoveNext())
return false;
m = _regex.Match((_objEnumerator.Current).Text); // (_objEnumerator.Current).Text хранит текст файла, ошибка падает в этой строчке
}
if (m.Success)
{
// код выдающий результат
return true;
}
else
{
_current = null;
return MoveNext();
}
}
}
Answer:
The problem, I think, is precisely because of the recursion (depth 2700, this is a lot). In .NET, starting with, it seems, 4.5, they switched to a new JIT compiler that can detect tail recursion, and make a tailcall
call instead of a normal recursive call. As a result, the stack is not clogged.
Since the presence of tail recursion optimization is not guaranteed by the language (and does not seem to work on 32-bit targets so far), the recursive implementation is a bug in the code. Rewrite your function iteratively.
Clarification: I tried code similar to yours and it doesn't generate tail call in IL code in VS 2015/.NET 4.5/x64/Release. So that may not be your problem. I'll try to investigate the cause further.
Further investigation: In the current version of .NET (4.5), tail recursion does not require the tail
IL prefix. Example: here is the code
class Program
{
[MethodImpl(MethodImplOptions.NoOptimization)]
static void Main(string[] args)
{
var t = new Program();
t.f(1);
Console.ReadKey(); // здесь можно приаттачить отладчик
t.f(100000000);
}
int f(int iterNo)
{
new DateTime(2017, 2, 3); // увеличим размер кода функции, чтобы сделать
// хвостовую оптимизацию привлекательной для JIT
if (iterNo == 0)
return 0;
else
return f(iterNo - 1);
}
}
generates the following IL, without the .tail
prefix:
.method private hidebysig instance int32
f(int32 iterNo) cil managed
{
// Code size 28 (0x1c)
.maxstack 8
//000021:
//000022: int f(int iterNo)
//000023: {
//000024: new DateTime(2017, 2, 3); // увеличим размер кода функции, чтобы сделать
IL_0000: ldc.i4 0x7e1
IL_0005: ldc.i4.2
IL_0006: ldc.i4.3
IL_0007: newobj instance void [mscorlib]System.DateTime::.ctor(int32,
int32,
int32)
IL_000c: pop
//000025: // хвостовую оптимизацию привлекательной для JIT
//000026: if (iterNo == 0)
IL_000d: ldarg.1
IL_000e: brtrue.s IL_0012
//000027: return 0;
IL_0010: ldc.i4.0
IL_0011: ret
//000028: else
//000029: return f(iterNo - 1);
IL_0012: ldarg.0
IL_0013: ldarg.1
IL_0014: ldc.i4.1
IL_0015: sub
IL_0016: call instance int32 Tailcall.Program::f(int32)
IL_001b: ret
} // end of method Program::f
However, the native code for f
looks like this:
24: new DateTime(2017, 2, 3); // увеличим размер кода функции, чтобы сделать
push rdi
push rsi
sub rsp,28h
mov rdi,rcx
mov esi,edx
mov ecx,7E1h
mov edx,2
mov r8d,3
call 000007FEF1361730
25: // хвостовую оптимизацию привлекательной для JIT
26: if (iterNo == 0)
test esi,esi
jne 000007FE932A053D
27: return 0;
xor eax,eax
add rsp,28h
pop rsi
pop rdi
ret
lea edx,[rsi-1] // вычли 1
mov rcx,rdi
mov rax,7FE932A0080h
add rsp,28h // очистили фрейм
pop rsi
pop rdi
jmp rax // переход вместо возврата
This means that tail recursion can be applied at the JIT level , without the involvement of an IL compiler. So the fact that .NET 4.5 has a new JIT compiler might well help.
In any case, the correct solution to the problem is to rewrite the code in an iterative way.