7 Commits
18 changed files with 734 additions and 97 deletions
+134
View File
@@ -1,5 +1,31 @@
# История: obdai.ru/receipt
## 2026-08-29: Android MVP camera pipeline
- Установлены пользовательские Android SDK 35, Build Tools 35.0.0 и Gradle 8.11.1.
- Добавлен Android-проект `android-app` с CameraX `ImageAnalysis`, ML Kit Text Recognition, RAM-only crop и multipart-клиентом `/receipt`.
- Исправлена конвертация `YUV_420_888` с учетом `rowStride`, `pixelStride` и поворота кадра.
- Старые кадры освобождаются при замене; запрещенные storage API в `app/src` не обнаружены.
- Добавлено масштабирование координат crop и JVM unit-тест `CropHelperTest` с Robolectric.
- Проверка `:app:testDebugUnitTest :app:assembleDebug` завершилась `BUILD SUCCESSFUL`.
- Версия Android-приложения повышена до `0.1.4`.
## 2026-08-29: Code review fixes
- Добавлено разрешение `INTERNET`; для OkHttp заданы connect/read/call timeouts.
- Аналитические Bitmap больше не рисуются поверх live preview на каждом кадре.
- YUV-конвертация учитывает `rowStride`, `pixelStride` и rotation; старые live-кадры освобождаются.
- Добавлен локальный privacy gate: признаки ФИО, пациента или даты блокируют отправку.
- ML Kit различает найденную зону, отсутствие текста и `PrivacyBlocked`.
- Unit-тесты и `assembleDebug` для версии `0.1.5` завершились успешно.
## 2026-08-29: Emulator check
- Создан AVD `receipt-api35` на Android 35 Google APIs x86_64.
- Запуск в текущем WSL невозможен: Android Emulator требует KVM с VMX/SVM, доступного аппаратного ускорения нет.
- `:app:testDebugUnitTest :app:assembleDebug` без эмулятора завершились `BUILD SUCCESSFUL`.
- Версия Android-приложения повышена до `0.1.6`.
## 2026-08-28
### Вопрос
@@ -622,3 +648,111 @@ nginx-маршруты без redirect и health `/receipt/health`. Старые
IP `127.0.0.1`, MIME `image/png`, размер 531101 байт, длину prompt 88 и
непустой usage JSON. Тестовый image-файл удалён после запроса; приложение
читает изображение в память и не пишет его на диск.
## 2026-08-31: Реализация backend-фиксов по плану v2
По команде пользователя выполнены изменения backend-компонентов и тестов.
### Изменения в recipe_service
- `recipe_service/metrics.py`:
- из `record()` удалён вызов `initialize()`;
- добавлена `count_since(client_ip, started_at_from)` для rate limiting.
- `recipe_service/app.py`:
- добавлен `ProxyFix(..., x_for=1, x_proto=1, x_host=1)`;
- сравнение токена переведено на `hmac.compare_digest`;
- введён единый финализатор `finalize(...)` вместо дублирования `record(...)`;
- `duration_ms` считается во всех ветках через `time.monotonic()`;
- добавлен rate limit `20` запросов/минута на IP (`429 too many requests`);
- зафиксирован контракт `502`:
`{"error":"upstream recognition failed","code":"upstream_error"}`;
- детали апстрима пишутся только в лог сервера с `request_id`.
- добавлен `recipe_service/test_app.py` (покрытие: `health`, `401`, `400`,
`415`, `413`, `200`, `502`-контракт, `429`).
### Изменения в gemini_proxy
- `gemini_proxy/app.py`: удалён `api_key_override`; ключ только из
`GEMINI_API_KEY`.
- `gemini_proxy/test_app.py`: добавлен тест, что `api_key_override` в форме
не даёт доступ без `GEMINI_API_KEY`.
### Изменения зависимостей
- выровнен root `requirements.txt` по version bounds с
`recipe_service/requirements.txt`:
- `Flask>=3.0,<4`
- `gunicorn>=21.2,<24`
- `requests>=2.31,<3`
### Проверки
- `py_compile` изменённых Python-файлов: успешно.
- `recipe_service`: `pytest -q` -> `8 passed`.
- `gemini_proxy`: `pytest -q` -> `4 passed`.
### Отдельно зафиксировано
Первый запуск тестов `recipe_service` дал `PermissionError` на `/var/lib/recipe`
при import-time `initialize()`. Исправлено в тесте ранней установкой
`RECIPE_METRICS_DB` в временный путь до импорта `app`.
## 2026-08-31: Безопасная оптимизация без смены поведения
По дополнительной команде пользователя выполнен пакет low-risk улучшений,
направленный на производительность и устойчивость, без изменения основного
контракта API.
### Изменения
- `recipe_service/metrics.py`:
- добавлен индекс
`idx_requests_client_ip_started_at ON requests(client_ip, started_at)`
для ускорения выборки rate limiting.
- `recipe_service/app.py`:
- ответ `429` унифицирован и дополнен стабильным полем
`code="rate_limited"` при сохранении `error="too many requests"`.
- `recipe_service/test_app.py`:
- обновлена проверка `429` с новым полем `code`;
- добавлен тест граничного случая лимитера (`19` запросов -> `200`);
- добавлен тест чтения последней записи в SQLite-метриках с проверкой
`status_code`, `duration_ms` и `error` после ветки `502`.
### Проверки
- `py_compile` изменённых Python-файлов: успешно.
- `recipe_service`: `pytest -q` -> `10 passed`.
- `gemini_proxy`: `pytest -q` -> `4 passed`.
### Вывод
Оптимизации применены без регрессий. Поведение успешного запроса, а также
статусы `400/401/413/415/502` сохранены; `429` дополнен машинным кодом
ошибки для стабильной клиентской обработки.
## 2026-08-31: Nginx-level rate limiting (основной лимитер)
По команде пользователя добавлен основной лимит запросов на уровне nginx,
при сохранении app-level fallback в `recipe_service/app.py`.
### Изменения конфигурации
- Добавлен новый файл `recipe_service/nginx-rate-limit-http.conf`:
- `limit_req_zone $binary_remote_addr zone=recipe_api_per_ip:10m rate=20r/m;`
- `limit_req_status 429;`
- файл предназначен для single-include внутри `http { ... }`.
- Обновлён `recipe_service/nginx-recipe.conf`:
- для `location = /recipe`, `location /recipe/`, `location = /receipt`,
`location /receipt/` добавлен
`limit_req zone=recipe_api_per_ip burst=5 nodelay;`.
### Результат
- Лимит теперь применяется единообразно для всех воркеров gunicorn на входе
nginx, а не только внутри отдельного процесса приложения.
- Python fallback-лимитер сохранён как защитный второй контур.
### Проверки
- `py_compile` изменённых Python-файлов: успешно.
- `recipe_service`: `pytest -q` -> `10 passed`.
+6 -3
View File
@@ -12,10 +12,11 @@ android {
applicationId = "ru.obdai.receipt"
minSdk = 26
targetSdk = 35
versionCode = 2
versionName = "0.1.1"
versionCode = 7
versionName = "0.1.6"
buildConfigField("String", "RECEIPT_API_TOKEN", "\"\"")
val apiToken = providers.environmentVariable("RECEIPT_API_TOKEN").orNull ?: ""
buildConfigField("String", "RECEIPT_API_TOKEN", "\"${apiToken.replace("\\", "\\\\").replace("\"", "\\\"")}\"")
buildConfigField("String", "RECEIPT_API_URL", "\"https://obdai.ru/receipt\"")
}
@@ -47,11 +48,13 @@ dependencies {
implementation("androidx.camera:camera-camera2:1.4.1")
implementation("androidx.camera:camera-lifecycle:1.4.1")
implementation("androidx.camera:camera-view:1.4.1")
implementation("com.google.mlkit:text-recognition:16.0.1")
implementation("com.squareup.okhttp3:okhttp:4.12.0")
implementation("org.jetbrains.kotlinx:kotlinx-coroutines-android:1.9.0")
implementation("org.jetbrains.kotlinx:kotlinx-serialization-json:1.7.3")
testImplementation("junit:junit:4.13.2")
testImplementation("org.robolectric:robolectric:4.14.1")
testImplementation("org.jetbrains.kotlinx:kotlinx-coroutines-test:1.9.0")
testImplementation("com.squareup.okhttp3:mockwebserver:4.12.0")
androidTestImplementation("androidx.test.ext:junit:1.2.1")
@@ -1,5 +1,6 @@
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
<uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.INTERNET" />
<application
android:allowBackup="false"
@@ -4,6 +4,7 @@ import android.Manifest
import android.content.pm.PackageManager
import android.os.Bundle
import android.graphics.Bitmap
import android.graphics.Rect
import androidx.activity.ComponentActivity
import androidx.activity.compose.setContent
import androidx.activity.result.contract.ActivityResultContracts
@@ -19,9 +20,11 @@ import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.padding
import androidx.compose.material3.CircularProgressIndicator
import androidx.compose.material3.Button
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.setValue
@@ -34,10 +37,20 @@ import androidx.compose.ui.unit.dp
import androidx.compose.ui.viewinterop.AndroidView
import androidx.core.content.ContextCompat
import ru.obdai.receipt.camera.CameraManager
import ru.obdai.receipt.crop.CropHelper
import ru.obdai.receipt.network.ApiClient
import ru.obdai.receipt.viewmodel.ReceiptViewModel
import ru.obdai.receipt.viewmodel.UiState
import androidx.lifecycle.ViewModelProvider
import androidx.lifecycle.ViewModel
import androidx.lifecycle.viewmodel.compose.viewModel
import java.util.concurrent.Executors
class MainActivity : ComponentActivity() {
private var latestBitmap by mutableStateOf<Bitmap?>(null)
private var capturedBitmap by mutableStateOf<Bitmap?>(null)
private var detectedBounds by mutableStateOf<Rect?>(null)
private var previewSize by mutableStateOf(android.util.Size(1, 1))
private val cameraExecutor = Executors.newSingleThreadExecutor()
private val cameraManager = CameraManager()
private val permissionLauncher = registerForActivityResult(ActivityResultContracts.RequestPermission()) { granted ->
@@ -46,7 +59,38 @@ class MainActivity : ComponentActivity() {
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContent { CameraScreen(latestBitmap, ::onPreviewReady) }
setContent {
val receiptViewModel: ReceiptViewModel = viewModel(
factory = object : ViewModelProvider.Factory {
@Suppress("UNCHECKED_CAST")
override fun <T : ViewModel> create(modelClass: Class<T>): T {
return ReceiptViewModel(
ApiClient(BuildConfig.RECEIPT_API_URL, BuildConfig.RECEIPT_API_TOKEN)
) as T
}
}
)
CameraScreen(
bitmap = latestBitmap,
capturedBitmap = capturedBitmap,
bounds = detectedBounds,
previewSize = previewSize,
state = receiptViewModel.state.collectAsState().value,
onPreviewReady = ::onPreviewReady,
onCapture = { capturedBitmap = latestBitmap },
onRecognize = { bitmap ->
receiptViewModel.detectZone(bitmap) { bounds ->
detectedBounds = bounds
if (bounds != null) {
receiptViewModel.recognize(
CropHelper.crop(bitmap, bounds),
"Recognize only medicine names, dosage, quantity and schedule. Return concise text."
)
}
}
}
)
}
if (ContextCompat.checkSelfPermission(this, Manifest.permission.CAMERA) != PackageManager.PERMISSION_GRANTED) {
permissionLauncher.launch(Manifest.permission.CAMERA)
}
@@ -55,6 +99,7 @@ class MainActivity : ComponentActivity() {
private fun onPreviewReady(view: PreviewView) {
if (previewView === view) return
previewView = view
view.post { previewSize = android.util.Size(view.width.coerceAtLeast(1), view.height.coerceAtLeast(1)) }
if (ContextCompat.checkSelfPermission(this, Manifest.permission.CAMERA) == PackageManager.PERMISSION_GRANTED) {
startCamera(view)
}
@@ -68,7 +113,13 @@ class MainActivity : ComponentActivity() {
val analysis = ImageAnalysis.Builder()
.setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
.build()
analysis.setAnalyzer(cameraExecutor, cameraManager.analyzer { bitmap -> latestBitmap = bitmap })
analysis.setAnalyzer(cameraExecutor, cameraManager.analyzer { bitmap ->
runOnUiThread {
val previous = latestBitmap
latestBitmap = bitmap
if (previous != null && previous !== capturedBitmap && !previous.isRecycled) previous.recycle()
}
})
provider.unbindAll()
preview.setSurfaceProvider(view.surfaceProvider)
provider.bindToLifecycle(this, CameraSelector.DEFAULT_BACK_CAMERA, preview, analysis)
@@ -79,13 +130,23 @@ class MainActivity : ComponentActivity() {
override fun onDestroy() {
latestBitmap?.let { if (!it.isRecycled) it.recycle() }
if (capturedBitmap !== latestBitmap) capturedBitmap?.let { if (!it.isRecycled) it.recycle() }
cameraExecutor.shutdown()
super.onDestroy()
}
}
@Composable
private fun CameraScreen(bitmap: Bitmap?, onPreviewReady: (PreviewView) -> Unit = {}) {
private fun CameraScreen(
bitmap: Bitmap?,
capturedBitmap: Bitmap?,
bounds: Rect?,
state: UiState,
previewSize: android.util.Size,
onPreviewReady: (PreviewView) -> Unit,
onCapture: () -> Unit,
onRecognize: (Bitmap) -> Unit
) {
Box(Modifier.fillMaxSize()) {
AndroidView(
factory = { context -> PreviewView(context).also(onPreviewReady) },
@@ -95,11 +156,47 @@ private fun CameraScreen(bitmap: Bitmap?, onPreviewReady: (PreviewView) -> Unit
modifier = Modifier.align(Alignment.BottomCenter).fillMaxWidth().padding(16.dp),
verticalArrangement = Arrangement.spacedBy(12.dp)
) {
Button(onClick = { /* Crop and recognize are wired by ReceiptViewModel. */ }, modifier = Modifier.fillMaxWidth()) {
Text("Распознать")
Button(onClick = onCapture, modifier = Modifier.fillMaxWidth()) {
Text("Зафиксировать кадр")
}
Button(
onClick = { capturedBitmap?.let(onRecognize) },
enabled = capturedBitmap != null && state !is UiState.Analyzing,
modifier = Modifier.fillMaxWidth()
) {
if (state is UiState.Analyzing) CircularProgressIndicator()
else Text(if (bounds == null) "Найти и распознать препараты" else "Распознать crop")
}
if (state is UiState.Error) Text(state.message, color = Color.Red)
if (state is UiState.PrivacyBlocked) {
Text("Обнаружены данные пациента. Отправка заблокирована.", color = Color.Red)
}
if (state is UiState.Result) Text(state.text, color = Color.Red, modifier = Modifier.padding(24.dp))
}
if (bitmap != null) ResultOverlay(bitmap, "")
if (capturedBitmap != null && bounds != null) {
CropOutline(
CropHelper.scaleToView(
bounds,
capturedBitmap.width,
capturedBitmap.height,
previewSize.width,
previewSize.height
)
)
}
if (state is UiState.Result) ResultOverlay(state.bitmap, state.text)
}
}
@Composable
private fun CropOutline(bounds: Rect) {
Canvas(Modifier.fillMaxSize()) {
drawRect(
color = Color.Green,
topLeft = androidx.compose.ui.geometry.Offset(bounds.left.toFloat(), bounds.top.toFloat()),
size = androidx.compose.ui.geometry.Size(bounds.width().toFloat(), bounds.height().toFloat()),
style = androidx.compose.ui.graphics.drawscope.Stroke(width = 4f)
)
}
}
@@ -108,5 +205,7 @@ private fun ResultOverlay(bitmap: Bitmap, text: String) {
Canvas(Modifier.fillMaxSize()) {
drawImage(bitmap.asImageBitmap(), dstSize = IntSize(size.width.toInt(), size.height.toInt()))
}
Text(text, color = Color.Red, modifier = Modifier.padding(24.dp))
if (text.isNotBlank()) {
Text(text, color = Color.Red, modifier = Modifier.padding(24.dp))
}
}
@@ -3,6 +3,7 @@ package ru.obdai.receipt.camera
import android.graphics.Bitmap
import android.graphics.BitmapFactory
import android.graphics.ImageFormat
import android.graphics.Matrix
import android.graphics.Rect
import android.graphics.YuvImage
import androidx.camera.core.ImageAnalysis
@@ -20,19 +21,42 @@ class CameraManager {
private fun ImageProxy.toBitmap(): Bitmap? {
if (format != ImageFormat.YUV_420_888 || planes.size < 3) return null
val y = planes[0].buffer
val u = planes[1].buffer
val v = planes[2].buffer
val ySize = y.remaining()
val uSize = u.remaining()
val vSize = v.remaining()
val nv21 = ByteArray(ySize + uSize + vSize)
y.get(nv21, 0, ySize)
v.get(nv21, ySize, vSize)
u.get(nv21, ySize + vSize, uSize)
val nv21 = ByteArray(width * height * 3 / 2)
copyPlane(planes[0], width, height, nv21, 0, 1)
copyPlane(planes[2], width / 2, height / 2, nv21, width * height, 2)
copyPlane(planes[1], width / 2, height / 2, nv21, width * height + 1, 2)
val jpeg = ByteArrayOutputStream()
YuvImage(nv21, ImageFormat.NV21, width, height, null)
.compressToJpeg(Rect(0, 0, width, height), 92, jpeg)
return BitmapFactory.decodeByteArray(jpeg.toByteArray(), 0, jpeg.size())
val decoded = BitmapFactory.decodeByteArray(jpeg.toByteArray(), 0, jpeg.size()) ?: return null
if (imageInfo.rotationDegrees == 0) return decoded
return Bitmap.createBitmap(
decoded,
0,
0,
decoded.width,
decoded.height,
Matrix().apply { postRotate(imageInfo.rotationDegrees.toFloat()) },
true
).also { if (it !== decoded) decoded.recycle() }
}
private fun copyPlane(
plane: ImageProxy.PlaneProxy,
planeWidth: Int,
planeHeight: Int,
output: ByteArray,
outputOffset: Int,
outputPixelStride: Int
) {
val buffer = plane.buffer.duplicate()
val rowStride = plane.rowStride
val pixelStride = plane.pixelStride
for (row in 0 until planeHeight) {
for (column in 0 until planeWidth) {
val sourceIndex = row * rowStride + column * pixelStride
output[outputOffset + row * planeWidth * outputPixelStride + column * outputPixelStride] = buffer.get(sourceIndex)
}
}
}
}
@@ -4,6 +4,16 @@ import android.graphics.Bitmap
import android.graphics.Rect
object CropHelper {
fun scaleToView(bounds: Rect, bitmapWidth: Int, bitmapHeight: Int, viewWidth: Int, viewHeight: Int): Rect {
require(bitmapWidth > 0 && bitmapHeight > 0 && viewWidth > 0 && viewHeight > 0)
return Rect(
bounds.left * viewWidth / bitmapWidth,
bounds.top * viewHeight / bitmapHeight,
bounds.right * viewWidth / bitmapWidth,
bounds.bottom * viewHeight / bitmapHeight
)
}
fun crop(source: Bitmap, bounds: Rect): Bitmap {
require(!source.isRecycled) { "Source bitmap is recycled" }
val left = bounds.left.coerceIn(0, source.width)
@@ -0,0 +1,51 @@
package ru.obdai.receipt.crop
import android.graphics.Bitmap
import android.graphics.Rect
import com.google.mlkit.vision.common.InputImage
import com.google.mlkit.vision.text.TextRecognition
import com.google.mlkit.vision.text.latin.TextRecognizerOptions
import kotlinx.coroutines.suspendCancellableCoroutine
import kotlin.coroutines.resume
import java.util.regex.Pattern
sealed interface ZoneDetection {
data class Found(val bounds: Rect) : ZoneDetection
data object NoText : ZoneDetection
data object PatientDataDetected : ZoneDetection
}
class MedicationZoneDetector {
private val recognizer = TextRecognition.getClient(TextRecognizerOptions.DEFAULT_OPTIONS)
suspend fun detect(bitmap: Bitmap): ZoneDetection = suspendCancellableCoroutine { continuation ->
recognizer.process(InputImage.fromBitmap(bitmap, 0))
.addOnSuccessListener { result ->
val text = result.text
if (containsPatientData(text)) {
continuation.resume(ZoneDetection.PatientDataDetected)
return@addOnSuccessListener
}
val blocks = result.textBlocks
.map { it.boundingBox }
.filterNotNull()
.filter { it.top > bitmap.height / 5 }
val bounds = blocks.reduceOrNull { first, next ->
Rect(first).apply { union(next) }
}
continuation.resume(bounds?.let(ZoneDetection::Found) ?: ZoneDetection.NoText)
}
.addOnFailureListener { continuation.resume(ZoneDetection.NoText) }
}
fun close() {
recognizer.close()
}
private fun containsPatientData(text: String): Boolean {
val normalized = text.lowercase()
val date = Pattern.compile("""\b\d{1,2}[./-]\d{1,2}[./-]\d{2,4}\b""").matcher(normalized).find()
val labels = listOf("ф.и.о", "фамилия", "имя", "отчество", "дата рождения", "пациент")
return date || labels.any(normalized::contains)
}
}
@@ -7,6 +7,7 @@ import okhttp3.MultipartBody
import okhttp3.OkHttpClient
import okhttp3.Request
import okhttp3.RequestBody.Companion.toRequestBody
import java.util.concurrent.TimeUnit
@Serializable
data class ReceiptResponse(val text: String? = null, val usage: Usage? = null)
@@ -17,7 +18,11 @@ data class Usage(val promptTokens: Int? = null, val candidatesTokens: Int? = nul
class ApiClient(
private val endpoint: String,
private val token: String,
private val client: OkHttpClient = OkHttpClient()
private val client: OkHttpClient = OkHttpClient.Builder()
.connectTimeout(15, TimeUnit.SECONDS)
.readTimeout(120, TimeUnit.SECONDS)
.callTimeout(150, TimeUnit.SECONDS)
.build()
) {
private val json = Json { ignoreUnknownKeys = true }
@@ -7,16 +7,22 @@ import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.launch
import ru.obdai.receipt.crop.MedicationZoneDetector
import ru.obdai.receipt.crop.ZoneDetection
import ru.obdai.receipt.network.ApiClient
sealed interface UiState {
data object Idle : UiState
data object Analyzing : UiState
data object PrivacyBlocked : UiState
data class Result(val bitmap: Bitmap, val text: String) : UiState
data class Error(val message: String) : UiState
}
class ReceiptViewModel(private val apiClient: ApiClient) : ViewModel() {
class ReceiptViewModel(
private val apiClient: ApiClient,
private val zoneDetector: MedicationZoneDetector = MedicationZoneDetector()
) : ViewModel() {
private val _state = MutableStateFlow<UiState>(UiState.Idle)
val state: StateFlow<UiState> = _state
@@ -35,17 +41,32 @@ class ReceiptViewModel(private val apiClient: ApiClient) : ViewModel() {
}
}
fun detectZone(bitmap: Bitmap, onDetected: (android.graphics.Rect?) -> Unit) {
viewModelScope.launch(Dispatchers.Default) {
when (val detection = zoneDetector.detect(bitmap)) {
is ZoneDetection.Found -> onDetected(detection.bounds)
ZoneDetection.PatientDataDetected -> {
_state.value = UiState.PrivacyBlocked
onDetected(null)
}
ZoneDetection.NoText -> onDetected(null)
}
}
}
override fun onCleared() {
(_state.value as? UiState.Result)?.bitmap?.let { bitmap ->
if (!bitmap.isRecycled) bitmap.recycle()
}
_state.value = UiState.Idle
zoneDetector.close()
super.onCleared()
}
private fun Bitmap.toJpegBytes(): ByteArray {
val output = java.io.ByteArrayOutputStream()
compress(Bitmap.CompressFormat.JPEG, 92, output)
return output.toByteArray()
return java.io.ByteArrayOutputStream().use { output ->
compress(Bitmap.CompressFormat.JPEG, 92, output)
output.toByteArray()
}
}
}
@@ -0,0 +1,16 @@
package ru.obdai.receipt.crop
import android.graphics.Rect
import org.junit.Assert.assertEquals
import org.junit.Test
import org.junit.runner.RunWith
import org.robolectric.RobolectricTestRunner
@RunWith(RobolectricTestRunner::class)
class CropHelperTest {
@Test
fun scalesBitmapBoundsToViewBounds() {
val result = CropHelper.scaleToView(Rect(100, 200, 500, 600), 1000, 1000, 500, 1000)
assertEquals(Rect(50, 200, 250, 600), result)
}
}
+1 -2
View File
@@ -25,7 +25,6 @@ async def recognize(
image: Annotated[UploadFile, File(...)],
prompt: Annotated[str, Form(...)],
generation_config: Annotated[str, Form()] = "{}",
api_key_override: Annotated[str | None, Form()] = None,
) -> dict:
if image.content_type not in ALLOWED_TYPES:
raise HTTPException(status_code=415, detail="Unsupported image type")
@@ -34,7 +33,7 @@ async def recognize(
if len(image_data) > MAX_IMAGE_BYTES:
raise HTTPException(status_code=413, detail="Image is too large")
api_key = api_key_override or os.getenv("GEMINI_API_KEY")
api_key = os.getenv("GEMINI_API_KEY")
if not api_key:
raise HTTPException(status_code=503, detail="Gemini is not configured")
+15
View File
@@ -28,4 +28,19 @@ def test_missing_key_returns_service_unavailable(monkeypatch) -> None:
files={"image": ("input.png", b"not-an-image", "image/png")},
data={"prompt": "test", "generation_config": "{}"},
)
assert response.status_code == 503
def test_rejects_when_only_override_provided(monkeypatch) -> None:
monkeypatch.delenv("GEMINI_API_KEY", raising=False)
client = TestClient(app)
response = client.post(
"/gemini",
files={"image": ("input.png", b"not-an-image", "image/png")},
data={
"prompt": "test",
"generation_config": "{}",
"api_key_override": "manual-key",
},
)
assert response.status_code == 503
+72 -65
View File
@@ -1,19 +1,24 @@
import json
import os
import hmac
import time
import requests
from flask import Flask, jsonify, request
from werkzeug.middleware.proxy_fix import ProxyFix
try:
from recipe_service.metrics import initialize, record, request_context, usage_json
from recipe_service.metrics import count_since, initialize, record, request_context, usage_json
except ModuleNotFoundError:
from metrics import initialize, record, request_context, usage_json
from metrics import count_since, initialize, record, request_context, usage_json
app = Flask(__name__)
app.wsgi_app = ProxyFix(app.wsgi_app, x_for=1, x_proto=1, x_host=1)
MAX_IMAGE_BYTES = 10 * 1024 * 1024
ALLOWED_TYPES = {"image/jpeg", "image/png", "image/webp"}
PROXY_URL = "http://127.0.0.1:8768/gemini"
RATE_LIMIT_REQUESTS_PER_MINUTE = 20
initialize()
@@ -31,7 +36,18 @@ def settings() -> dict:
def authorized() -> bool:
expected = os.environ.get("RECIPE_API_TOKEN")
authorization = request.headers.get("Authorization", "")
return bool(expected and authorization == f"Bearer {expected}")
return bool(expected and hmac.compare_digest(authorization, f"Bearer {expected}"))
def minute_start_utc(epoch_seconds: float) -> str:
return time.strftime("%Y-%m-%dT%H:%M:00Z", time.gmtime(epoch_seconds))
def is_rate_limited(client_ip: str, now_epoch: float) -> bool:
if not client_ip:
return False
window_start = minute_start_utc(now_epoch)
return count_since(client_ip=client_ip, started_at_from=window_start) >= RATE_LIMIT_REQUESTS_PER_MINUTE
@app.get("/health")
@@ -49,54 +65,58 @@ def recipe():
image_mime = None
image_bytes = None
prompt_chars = None
status_code = 500
response_bytes = None
usage = {}
error = None
if not authorized():
status_code = 401
error = "unauthorized"
response = jsonify(error=error)
record(request_id=request_id, started_at=started_at, client_ip=request.remote_addr,
user_agent=request.user_agent.string, method=request.method, path=request.path,
image_mime=image_mime, image_bytes=image_bytes, prompt_chars=prompt_chars,
status_code=status_code, duration_ms=int((__import__('time').monotonic() - started_monotonic) * 1000),
response_bytes=len(response.get_data()), usage_json=usage_json(usage), error=error)
client_ip = request.remote_addr
def finalize(response, status_code: int, error: str | None):
duration_ms = int((time.monotonic() - started_monotonic) * 1000)
response_bytes = len(response.get_data())
record(
request_id=request_id,
started_at=started_at,
client_ip=client_ip,
user_agent=request.user_agent.string,
method=request.method,
path=request.path,
image_mime=image_mime,
image_bytes=image_bytes,
prompt_chars=prompt_chars,
status_code=status_code,
duration_ms=duration_ms,
response_bytes=response_bytes,
usage_json=usage_json(usage),
error=error,
)
return response, status_code
if not authorized():
return finalize(jsonify(error="unauthorized"), 401, "unauthorized")
if is_rate_limited(client_ip=client_ip or "", now_epoch=time.time()):
return finalize(
jsonify(error="too many requests", code="rate_limited"),
429,
"rate_limited",
)
image = request.files.get("image")
prompt = request.form.get("prompt")
if image is None or not prompt:
status_code = 400
error = "image and prompt are required"
response = jsonify(error=error)
record(request_id=request_id, started_at=started_at, client_ip=request.remote_addr,
user_agent=request.user_agent.string, method=request.method, path=request.path,
status_code=status_code, duration_ms=0, response_bytes=len(response.get_data()),
usage_json=usage_json(usage), error=error)
return response, status_code
return finalize(jsonify(error="image and prompt are required"), 400, "image and prompt are required")
prompt_chars = len(prompt)
if image.mimetype not in ALLOWED_TYPES:
status_code = 415
error = "unsupported image type"
response = jsonify(error=error)
record(request_id=request_id, started_at=started_at, client_ip=request.remote_addr,
user_agent=request.user_agent.string, method=request.method, path=request.path,
image_mime=image.mimetype, prompt_chars=len(prompt), status_code=status_code,
duration_ms=0, response_bytes=len(response.get_data()), usage_json=usage_json(usage), error=error)
return response, status_code
image_mime = image.mimetype
return finalize(jsonify(error="unsupported image type"), 415, "unsupported image type")
image_data = image.read(MAX_IMAGE_BYTES + 1)
if len(image_data) > MAX_IMAGE_BYTES:
status_code = 413
error = "image is too large"
response = jsonify(error=error)
record(request_id=request_id, started_at=started_at, client_ip=request.remote_addr,
user_agent=request.user_agent.string, method=request.method, path=request.path,
image_mime=image.mimetype, image_bytes=len(image_data), prompt_chars=len(prompt),
status_code=status_code, duration_ms=0, response_bytes=len(response.get_data()),
usage_json=usage_json(usage), error=error)
return response, status_code
image_mime = image.mimetype
image_bytes = len(image_data)
prompt_chars = len(prompt)
if len(image_data) > MAX_IMAGE_BYTES:
return finalize(jsonify(error="image is too large"), 413, "image is too large")
try:
config = settings()
response = requests.post(
@@ -107,34 +127,21 @@ def recipe():
)
except (requests.RequestException, RuntimeError) as exc:
error = str(exc) if isinstance(exc, RuntimeError) else "Gemini unavailable"
status_code = 503
response = jsonify(error=error)
record(request_id=request_id, started_at=started_at, client_ip=request.remote_addr,
user_agent=request.user_agent.string, method=request.method, path=request.path,
image_mime=image_mime, image_bytes=image_bytes, prompt_chars=prompt_chars,
status_code=status_code, duration_ms=0, response_bytes=len(response.get_data()),
usage_json=usage_json(usage), error=error)
return response, status_code
return finalize(jsonify(error=error), 503, error)
if response.status_code != 200:
try:
detail = response.json().get("error", {}).get("message", "Gemini request failed")
except ValueError:
detail = "Gemini request failed"
error = detail
status_code = 502
response = jsonify(error=error)
record(request_id=request_id, started_at=started_at, client_ip=request.remote_addr,
user_agent=request.user_agent.string, method=request.method, path=request.path,
image_mime=image_mime, image_bytes=image_bytes, prompt_chars=prompt_chars,
status_code=status_code, duration_ms=0, response_bytes=len(response.get_data()),
usage_json=usage_json(usage), error=error)
return response, status_code
app.logger.warning("upstream_failure request_id=%s detail=%s", request_id, detail)
return finalize(
jsonify(error="upstream recognition failed", code="upstream_error"),
502,
"upstream_error",
)
data = response.json()
usage = data.get("usage", {})
result = jsonify(text=data.get("text"), usage=usage)
record(request_id=request_id, started_at=started_at, client_ip=request.remote_addr,
user_agent=request.user_agent.string, method=request.method, path=request.path,
image_mime=image_mime, image_bytes=image_bytes, prompt_chars=prompt_chars,
status_code=200, duration_ms=0, response_bytes=len(result.get_data()),
usage_json=usage_json(usage), error=None)
return result
return finalize(result, 200, None)
+18 -1
View File
@@ -37,10 +37,12 @@ def initialize() -> None:
""")
connection.execute("CREATE INDEX IF NOT EXISTS idx_requests_started_at ON requests(started_at)")
connection.execute("CREATE INDEX IF NOT EXISTS idx_requests_status_code ON requests(status_code)")
connection.execute(
"CREATE INDEX IF NOT EXISTS idx_requests_client_ip_started_at ON requests(client_ip, started_at)"
)
def record(**values) -> None:
initialize()
columns = [
"request_id", "started_at", "client_ip", "user_agent", "method",
"path", "image_mime", "image_bytes", "prompt_chars", "status_code",
@@ -54,6 +56,21 @@ def record(**values) -> None:
)
def count_since(client_ip: str, started_at_from: str) -> int:
with sqlite3.connect(db_path()) as connection:
row = connection.execute(
"""
SELECT COUNT(*)
FROM requests
WHERE client_ip = ?
AND started_at >= ?
AND path IN ('/receipt', '/receipt/', '/recipe', '/recipe/')
""",
(client_ip, started_at_from),
).fetchone()
return int(row[0] if row else 0)
def request_context() -> tuple[str, str, float]:
return str(uuid.uuid4()), time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()), time.monotonic()
@@ -0,0 +1,8 @@
# Include this file once inside nginx `http { ... }` block.
# Example: include /etc/nginx/conf.d/recipe-rate-limit-http.conf;
# Per-client limit for recipe/receipt API requests.
limit_req_zone $binary_remote_addr zone=recipe_api_per_ip:10m rate=20r/m;
# Return 429 for throttled requests.
limit_req_status 429;
+4
View File
@@ -1,4 +1,5 @@
location = /recipe {
limit_req zone=recipe_api_per_ip burst=5 nodelay;
proxy_pass http://127.0.0.1:8770/recipe;
proxy_http_version 1.1;
proxy_set_header Host $host;
@@ -12,6 +13,7 @@ location = /recipe {
}
location /recipe/ {
limit_req zone=recipe_api_per_ip burst=5 nodelay;
proxy_pass http://127.0.0.1:8770/recipe/;
proxy_http_version 1.1;
proxy_set_header Host $host;
@@ -25,6 +27,7 @@ location /recipe/ {
}
location = /receipt {
limit_req zone=recipe_api_per_ip burst=5 nodelay;
proxy_pass http://127.0.0.1:8770/receipt;
proxy_http_version 1.1;
proxy_set_header Host $host;
@@ -38,6 +41,7 @@ location = /receipt {
}
location /receipt/ {
limit_req zone=recipe_api_per_ip burst=5 nodelay;
proxy_pass http://127.0.0.1:8770/receipt/;
proxy_http_version 1.1;
proxy_set_header Host $host;
+223
View File
@@ -0,0 +1,223 @@
import io
import json
import os
import sqlite3
import tempfile
import requests
os.environ.setdefault(
"RECIPE_METRICS_DB",
os.path.join(tempfile.gettempdir(), "recipe-service-tests-metrics.sqlite3"),
)
from app import app
class MockResponse:
def __init__(self, status_code: int, payload: dict | None = None):
self.status_code = status_code
self._payload = payload or {}
def json(self) -> dict:
return self._payload
def auth_header() -> dict[str, str]:
return {"Authorization": "Bearer test-token"}
def make_image(content: bytes = b"img") -> tuple[io.BytesIO, str, str]:
return io.BytesIO(content), "sample.png", "image/png"
def test_health() -> None:
client = app.test_client()
response = client.get("/health")
assert response.status_code == 200
assert response.get_json() == {"status": "ok"}
def test_requires_authorization(monkeypatch) -> None:
monkeypatch.setenv("RECIPE_API_TOKEN", "test-token")
client = app.test_client()
response = client.post("/receipt")
assert response.status_code == 401
assert response.get_json() == {"error": "unauthorized"}
def test_missing_image_or_prompt(monkeypatch) -> None:
monkeypatch.setenv("RECIPE_API_TOKEN", "test-token")
client = app.test_client()
response = client.post("/receipt", headers=auth_header())
assert response.status_code == 400
assert response.get_json() == {"error": "image and prompt are required"}
def test_unsupported_type(monkeypatch) -> None:
monkeypatch.setenv("RECIPE_API_TOKEN", "test-token")
client = app.test_client()
response = client.post(
"/receipt",
headers=auth_header(),
data={
"prompt": "p",
"image": (io.BytesIO(b"x"), "bad.txt", "text/plain"),
},
content_type="multipart/form-data",
)
assert response.status_code == 415
assert response.get_json() == {"error": "unsupported image type"}
def test_image_too_large(monkeypatch) -> None:
monkeypatch.setenv("RECIPE_API_TOKEN", "test-token")
client = app.test_client()
payload = b"a" * (10 * 1024 * 1024 + 1)
response = client.post(
"/receipt",
headers=auth_header(),
data={
"prompt": "p",
"image": (io.BytesIO(payload), "big.png", "image/png"),
},
content_type="multipart/form-data",
)
assert response.status_code == 413
assert response.get_json() == {"error": "image is too large"}
def test_success(monkeypatch) -> None:
monkeypatch.setenv("RECIPE_API_TOKEN", "test-token")
def fake_post(*args, **kwargs):
return MockResponse(200, {"text": "ok", "usage": {"totalTokens": 10}})
monkeypatch.setattr(requests, "post", fake_post)
client = app.test_client()
response = client.post(
"/receipt",
headers=auth_header(),
data={
"prompt": "p",
"image": make_image(),
},
content_type="multipart/form-data",
)
assert response.status_code == 200
assert response.get_json() == {"text": "ok", "usage": {"totalTokens": 10}}
def test_upstream_502_contract(monkeypatch) -> None:
monkeypatch.setenv("RECIPE_API_TOKEN", "test-token")
def fake_post(*args, **kwargs):
return MockResponse(500, {"error": {"message": "provider detail"}})
monkeypatch.setattr(requests, "post", fake_post)
client = app.test_client()
response = client.post(
"/receipt",
headers=auth_header(),
data={
"prompt": "p",
"image": make_image(),
},
content_type="multipart/form-data",
)
assert response.status_code == 502
assert response.get_json() == {
"error": "upstream recognition failed",
"code": "upstream_error",
}
def test_rate_limit_returns_429(monkeypatch) -> None:
monkeypatch.setenv("RECIPE_API_TOKEN", "test-token")
def fake_count_since(client_ip: str, started_at_from: str) -> int:
return 20
monkeypatch.setattr("app.count_since", fake_count_since)
client = app.test_client()
response = client.post(
"/receipt",
headers=auth_header(),
data={
"prompt": "p",
"image": make_image(),
},
content_type="multipart/form-data",
environ_base={"REMOTE_ADDR": "198.51.100.10"},
)
assert response.status_code == 429
assert response.get_json() == {"error": "too many requests", "code": "rate_limited"}
def test_rate_limit_allows_below_threshold(monkeypatch) -> None:
monkeypatch.setenv("RECIPE_API_TOKEN", "test-token")
def fake_count_since(client_ip: str, started_at_from: str) -> int:
return 19
def fake_post(*args, **kwargs):
return MockResponse(200, {"text": "ok", "usage": {}})
monkeypatch.setattr("app.count_since", fake_count_since)
monkeypatch.setattr(requests, "post", fake_post)
client = app.test_client()
response = client.post(
"/receipt",
headers=auth_header(),
data={
"prompt": "p",
"image": make_image(),
},
content_type="multipart/form-data",
environ_base={"REMOTE_ADDR": "198.51.100.11"},
)
assert response.status_code == 200
def test_metrics_record_duration_and_status(monkeypatch) -> None:
db_file = os.path.join(tempfile.gettempdir(), "recipe-service-tests-metrics-duration.sqlite3")
if os.path.exists(db_file):
os.remove(db_file)
monkeypatch.setenv("RECIPE_METRICS_DB", db_file)
monkeypatch.setenv("RECIPE_API_TOKEN", "test-token")
from metrics import initialize
initialize()
def fake_post(*args, **kwargs):
return MockResponse(500, {"error": {"message": "provider detail"}})
monkeypatch.setattr(requests, "post", fake_post)
client = app.test_client()
response = client.post(
"/receipt",
headers=auth_header(),
data={
"prompt": "p",
"image": make_image(),
},
content_type="multipart/form-data",
)
assert response.status_code == 502
connection = sqlite3.connect(db_file)
try:
row = connection.execute(
"SELECT status_code, duration_ms, error FROM requests ORDER BY rowid DESC LIMIT 1"
).fetchone()
finally:
connection.close()
assert row is not None
status_code, duration_ms, error = row
assert status_code == 502
assert duration_ms >= 0
assert error == "upstream_error"
+3 -3
View File
@@ -1,3 +1,3 @@
Flask>=3.0
gunicorn>=21.2
requests>=2.31
Flask>=3.0,<4
gunicorn>=21.2,<24
requests>=2.31,<3