projects / openwrt / staging / stintel.git / blob
? search:


d5babb1fea79e66bf42f45a1f9928612ec59c03c
[openwrt/staging/stintel.git] /
1 From: Md Sadre Alam <quic_mdalam@quicinc.com>
2 To: <broonie@kernel.org>, <robh@kernel.org>, <krzk+dt@kernel.org>,
3 <conor+dt@kernel.org>, <andersson@kernel.org>,
4 <konradybcio@kernel.org>, <miquel.raynal@bootlin.com>,
5 <richard@nod.at>, <vigneshr@ti.com>,
6 <manivannan.sadhasivam@linaro.org>,
7 <linux-arm-msm@vger.kernel.org>, <linux-spi@vger.kernel.org>,
8 <devicetree@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
9 <linux-mtd@lists.infradead.org>
10 Cc: <quic_srichara@quicinc.com>, <quic_varada@quicinc.com>,
11 <quic_mdalam@quicinc.com>
12 Subject: [PATCH v14 6/8] spi: spi-qpic: add driver for QCOM SPI NAND flash Interface
13 Date: Wed, 20 Nov 2024 14:45:04 +0530 [thread overview]
14 Message-ID: <20241120091507.1404368-7-quic_mdalam@quicinc.com> (raw)
15 In-Reply-To: <20241120091507.1404368-1-quic_mdalam@quicinc.com>
16
17 This driver implements support for the SPI-NAND mode of QCOM NAND Flash
18 Interface as a SPI-MEM controller with pipelined ECC capability.
19
20 Co-developed-by: Sricharan Ramabadhran <quic_srichara@quicinc.com>
21 Signed-off-by: Sricharan Ramabadhran <quic_srichara@quicinc.com>
22 Co-developed-by: Varadarajan Narayanan <quic_varada@quicinc.com>
23 Signed-off-by: Varadarajan Narayanan <quic_varada@quicinc.com>
24 Signed-off-by: Md Sadre Alam <quic_mdalam@quicinc.com>
25 ---
26
27 Change in [v14]
28
29 * No Change
30
31 Change in [v13]
32
33 * Changed return type of qcom_spi_cmd_mapping() from u32 to
34 int to fix the kernel test bot warning
35 * Changed type of variable cmd in qcom_spi_write_page() from u32
36 to int
37 * Removed unused variable s_op from qcom_spi_write_page()
38 * Updated return value variable type from u32 to int in
39 qcom_spi_send_cmdaddr()
40
41 Change in [v12]
42
43 * Added obj-$(CONFIG_SPI_QPIC_SNAND) += qpic_common.o in Makefile
44 to build qpic_common.c based on CONFIG_SPI_QPIC_SNAND
45
46 Change in [v11]
47
48 * Fixed build error reported by kernel test bot
49 * Changed "depends on MTD" to "select MTD" in
50 drivers/spi/Kconfig file
51
52 Change in [v10]
53
54 * Fixed compilation warnings reported by kernel test robot.
55 * Added depends on CONFIG_MTD
56 * removed extra bracket from statement if (i == (num_cw - 1)) in
57 qcom_spi_program_raw() api.
58
59 Change in [v9]
60
61 * Changed data type of addr1, addr2, cmd, to __le32 in qpic_spi_nand
62 structure
63 * In qcom_spi_set_read_loc_first() api added cpu_to_le32() macro to fix
64 compilation warning
65 * In qcom_spi_set_read_loc_last() api added cpu_to_le32() macro to fix
66 compilation warning
67 * In qcom_spi_init() api added cpu_to_le32() macro to fix compilation
68 warning
69 * In qcom_spi_ecc_init_ctx_pipelined() api removed unused variables
70 reqs, user, step_size, strength and added cpu_to_le32() macro as well
71 to fix compilation warning
72 * In qcom_spi_read_last_cw() api added cpu_to_le32() macro to fix compilation
73 warning
74 * In qcom_spi_check_error() api added cpu_to_le32() macro to fix compilation
75 warning
76 * In qcom_spi_read_page_ecc() api added cpu_to_le32() macro to fix compilation
77 warning
78 * In qcom_spi_read_page_oob() api added cpu_to_le32() macro to fix compilation
79 warning
80 * In qcom_spi_program_raw() api added cpu_to_le32() macro to fix compilation
81 warning
82 * In qcom_spi_program_ecc() api added cpu_to_le32() macro to fix compilation
83 warning
84 * In qcom_spi_program_oob() api added cpu_to_le32() macro to fix compilation
85 warning
86 * In qcom_spi_send_cmdaddr() api added cpu_to_le32() macro to fix compilation
87 warning
88 * In qcom_spi_io_op() api added cpu_to_le32() macro to fix compilation
89 warning
90
91 Change in [v8]
92
93 * Included "bitfield.h" file to /spi-qpic-snand.c
94 to fix compilation warning reported by kernel test robot
95 * Removed unused variable "steps" in
96 qcom_spi_ecc_init_ctx_pipelined() to fix compilation warning
97
98 Change in [v7]
99
100 * Added read_oob() and write_oob() api
101
102 * Handled offset value for oob layout
103
104 * Made CONFIG_SPI_QPIC_SNAND as bool
105
106 * Added macro ecceng_to_qspi()
107
108 * Added FIELD_PREP() Macro in spi init
109
110 * Added else condition in
111 qcom_spi_ecc_finish_io_req_pipelined()
112 for corrected ecc
113
114 * Handled multiple error condition for api
115 qcom_spi_cmd_mapping()
116
117 * Fix typo for printing debug message
118
119 Change in [v6]
120
121 * Added separate qpic_spi_nand{...} struct
122
123 * moved qpic_ecc and qcom_ecc_stats struct to
124 spi-qpic-snand.c file, since its spi nand
125 specific
126
127 * Added FIELD_PREP() and GENMASK() macro
128
129 * Removed rawnand.h and partition.h from
130 spi-qpic-snand.c
131
132 * Removed oob_buff assignment form
133 qcom_spi_write_page_cache
134
135 * Added qcom_nand_unalloc() in remove() path
136
137 * Fixes all all comments
138
139 Change in [v5]
140
141 * Added raw_read() and raw_write() api
142
143 * Updated commit message
144
145 * Removed register indirection
146
147 * Added qcom_spi_ prefix to all the api
148
149 * Removed snand_set_reg() api.
150
151 * Fixed nandbiterr issue
152
153 * Removed hardcoded num_cw and made it variable
154
155 * Removed hardcoded value for mtd pagesize
156
157 * Added -ENOSUPPORT in cmd mapping for unsupported
158 commands
159
160 * Replace if..else with switch..case statement
161
162 Change in [v4]
163
164 * No change
165
166 Change in [v3]
167
168 * Set SPI_QPIC_SNAND to n and added COMPILE_TEST in Kconfig
169
170 * Made driver name sorted in Make file
171
172 * Made comment like c++
173
174 * Changed macro to functions, snandc_set_read_loc_last()
175 and snandc_set_read_loc_first()
176
177 * Added error handling in snandc_set_reg()
178
179 * Changed into normal conditional statement for
180 return snandc->ecc_stats.failed ? -EBADMSG :
181 snandc->ecc_stats.bitflips;
182
183 * Remove cast of wbuf in qpic_snand_program_execute()
184 function
185
186 * Made num_cw variable instead hardcoded value
187
188 * changed if..else condition of function qpic_snand_io_op()
189 to switch..case statement
190
191 * Added __devm_spi_alloc_controller() api instead of
192 devm_spi_alloc_master()
193
194 * Disabling clock in remove path
195
196 Change in [v2]
197
198 * Added initial support for SPI-NAND driver
199
200 Change in [v1]
201
202 * Added RFC patch for design review
203
204 drivers/mtd/nand/Makefile | 4 +
205 drivers/spi/Kconfig | 9 +
206 drivers/spi/Makefile | 1 +
207 drivers/spi/spi-qpic-snand.c | 1633 ++++++++++++++++++++++++++
208 include/linux/mtd/nand-qpic-common.h | 7 +
209 5 files changed, 1654 insertions(+)
210 create mode 100644 drivers/spi/spi-qpic-snand.c
211
212 diff --git a/drivers/mtd/nand/Makefile b/drivers/mtd/nand/Makefile
213 index da1586a36574..db516a45f0c5 100644
214 --- a/drivers/mtd/nand/Makefile
215 +++ b/drivers/mtd/nand/Makefile
216 @@ -3,7 +3,11 @@
217 obj-$(CONFIG_MTD_NAND_CORE) += nandcore.o
218 obj-$(CONFIG_MTD_NAND_ECC_MEDIATEK) += ecc-mtk.o
219 obj-$(CONFIG_MTD_NAND_MTK_BMT) += mtk_bmt.o mtk_bmt_v2.o mtk_bmt_bbt.o mtk_bmt_nmbm.o
220 +ifeq ($(CONFIG_SPI_QPIC_SNAND),y)
221 +obj-$(CONFIG_SPI_QPIC_SNAND) += qpic_common.o
222 +else
223 obj-$(CONFIG_MTD_NAND_QCOM) += qpic_common.o
224 +endif
225 obj-y += onenand/
226 obj-y += raw/
227 obj-y += spi/
228 diff --git a/drivers/spi/Kconfig b/drivers/spi/Kconfig
229 index f51f9466e518..1aaf93964429 100644
230 --- a/drivers/spi/Kconfig
231 +++ b/drivers/spi/Kconfig
232 @@ -920,6 +920,15 @@ config SPI_QCOM_QSPI
233 help
234 QSPI(Quad SPI) driver for Qualcomm QSPI controller.
235
236 +config SPI_QPIC_SNAND
237 + bool "QPIC SNAND controller"
238 + depends on ARCH_QCOM || COMPILE_TEST
239 + select MTD
240 + help
241 + QPIC_SNAND (QPIC SPI NAND) driver for Qualcomm QPIC controller.
242 + QPIC controller supports both parallel nand and serial nand.
243 + This config will enable serial nand driver for QPIC controller.
244 +
245 config SPI_QUP
246 tristate "Qualcomm SPI controller with QUP interface"
247 depends on ARCH_QCOM || COMPILE_TEST
248 diff --git a/drivers/spi/Makefile b/drivers/spi/Makefile
249 index aea5e54de195..3309b7bb2445 100644
250 --- a/drivers/spi/Makefile
251 +++ b/drivers/spi/Makefile
252 @@ -115,6 +115,7 @@ obj-$(CONFIG_SPI_PXA2XX) += spi-pxa2xx-platform.o
253 obj-$(CONFIG_SPI_PXA2XX_PCI) += spi-pxa2xx-pci.o
254 obj-$(CONFIG_SPI_QCOM_GENI) += spi-geni-qcom.o
255 obj-$(CONFIG_SPI_QCOM_QSPI) += spi-qcom-qspi.o
256 +obj-$(CONFIG_SPI_QPIC_SNAND) += spi-qpic-snand.o
257 obj-$(CONFIG_SPI_QUP) += spi-qup.o
258 obj-$(CONFIG_SPI_ROCKCHIP) += spi-rockchip.o
259 obj-$(CONFIG_SPI_ROCKCHIP_SFC) += spi-rockchip-sfc.o
260 diff --git a/drivers/spi/spi-qpic-snand.c b/drivers/spi/spi-qpic-snand.c
261 new file mode 100644
262 index 000000000000..1ba562a9369e
263 --- /dev/null
264 +++ b/drivers/spi/spi-qpic-snand.c
265 @@ -0,0 +1,1633 @@
266 +/*
267 + * SPDX-License-Identifier: GPL-2.0
268 + *
269 + * Copyright (c) 2023, Qualcomm Innovation Center, Inc. All rights reserved.
270 + *
271 + * Authors:
272 + * Md Sadre Alam <quic_mdalam@quicinc.com>
273 + * Sricharan R <quic_srichara@quicinc.com>
274 + * Varadarajan Narayanan <quic_varada@quicinc.com>
275 + */
276 +#include <linux/bitops.h>
277 +#include <linux/clk.h>
278 +#include <linux/delay.h>
279 +#include <linux/dmaengine.h>
280 +#include <linux/dma-mapping.h>
281 +#include <linux/dma/qcom_adm.h>
282 +#include <linux/dma/qcom_bam_dma.h>
283 +#include <linux/module.h>
284 +#include <linux/of.h>
285 +#include <linux/platform_device.h>
286 +#include <linux/slab.h>
287 +#include <linux/mtd/nand-qpic-common.h>
288 +#include <linux/mtd/spinand.h>
289 +#include <linux/bitfield.h>
290 +
291 +#define NAND_FLASH_SPI_CFG 0xc0
292 +#define NAND_NUM_ADDR_CYCLES 0xc4
293 +#define NAND_BUSY_CHECK_WAIT_CNT 0xc8
294 +#define NAND_FLASH_FEATURES 0xf64
295 +
296 +/* QSPI NAND config reg bits */
297 +#define LOAD_CLK_CNTR_INIT_EN BIT(28)
298 +#define CLK_CNTR_INIT_VAL_VEC 0x924
299 +#define CLK_CNTR_INIT_VAL_VEC_MASK GENMASK(27, 16)
300 +#define FEA_STATUS_DEV_ADDR 0xc0
301 +#define FEA_STATUS_DEV_ADDR_MASK GENMASK(15, 8)
302 +#define SPI_CFG BIT(0)
303 +#define SPI_NUM_ADDR 0xDA4DB
304 +#define SPI_WAIT_CNT 0x10
305 +#define QPIC_QSPI_NUM_CS 1
306 +#define SPI_TRANSFER_MODE_x1 BIT(29)
307 +#define SPI_TRANSFER_MODE_x4 (3 << 29)
308 +#define SPI_WP BIT(28)
309 +#define SPI_HOLD BIT(27)
310 +#define QPIC_SET_FEATURE BIT(31)
311 +
312 +#define SPINAND_RESET 0xff
313 +#define SPINAND_READID 0x9f
314 +#define SPINAND_GET_FEATURE 0x0f
315 +#define SPINAND_SET_FEATURE 0x1f
316 +#define SPINAND_READ 0x13
317 +#define SPINAND_ERASE 0xd8
318 +#define SPINAND_WRITE_EN 0x06
319 +#define SPINAND_PROGRAM_EXECUTE 0x10
320 +#define SPINAND_PROGRAM_LOAD 0x84
321 +
322 +#define ACC_FEATURE 0xe
323 +#define BAD_BLOCK_MARKER_SIZE 0x2
324 +#define OOB_BUF_SIZE 128
325 +#define ecceng_to_qspi(eng) container_of(eng, struct qpic_spi_nand, ecc_eng)
326 +struct qpic_snand_op {
327 + u32 cmd_reg;
328 + u32 addr1_reg;
329 + u32 addr2_reg;
330 +};
331 +
332 +struct snandc_read_status {
333 + __le32 snandc_flash;
334 + __le32 snandc_buffer;
335 + __le32 snandc_erased_cw;
336 +};
337 +
338 +/*
339 + * ECC state struct
340 + * @corrected: ECC corrected
341 + * @bitflips: Max bit flip
342 + * @failed: ECC failed
343 + */
344 +struct qcom_ecc_stats {
345 + u32 corrected;
346 + u32 bitflips;
347 + u32 failed;
348 +};
349 +
350 +struct qpic_ecc {
351 + struct device *dev;
352 + int ecc_bytes_hw;
353 + int spare_bytes;
354 + int bbm_size;
355 + int ecc_mode;
356 + int bytes;
357 + int steps;
358 + int step_size;
359 + int strength;
360 + int cw_size;
361 + int cw_data;
362 + u32 cfg0;
363 + u32 cfg1;
364 + u32 cfg0_raw;
365 + u32 cfg1_raw;
366 + u32 ecc_buf_cfg;
367 + u32 ecc_bch_cfg;
368 + u32 clrflashstatus;
369 + u32 clrreadstatus;
370 + bool bch_enabled;
371 +};
372 +
373 +struct qpic_spi_nand {
374 + struct qcom_nand_controller *snandc;
375 + struct spi_controller *ctlr;
376 + struct mtd_info *mtd;
377 + struct clk *iomacro_clk;
378 + struct qpic_ecc *ecc;
379 + struct qcom_ecc_stats ecc_stats;
380 + struct nand_ecc_engine ecc_eng;
381 + u8 *data_buf;
382 + u8 *oob_buf;
383 + u32 wlen;
384 + __le32 addr1;
385 + __le32 addr2;
386 + __le32 cmd;
387 + u32 num_cw;
388 + bool oob_rw;
389 + bool page_rw;
390 + bool raw_rw;
391 +};
392 +
393 +static void qcom_spi_set_read_loc_first(struct qcom_nand_controller *snandc,
394 + int reg, int cw_offset, int read_size,
395 + int is_last_read_loc)
396 +{
397 + __le32 locreg_val;
398 + u32 val = (((cw_offset) << READ_LOCATION_OFFSET) |
399 + ((read_size) << READ_LOCATION_SIZE) | ((is_last_read_loc)
400 + << READ_LOCATION_LAST));
401 +
402 + locreg_val = cpu_to_le32(val);
403 +
404 + if (reg == NAND_READ_LOCATION_0)
405 + snandc->regs->read_location0 = locreg_val;
406 + else if (reg == NAND_READ_LOCATION_1)
407 + snandc->regs->read_location1 = locreg_val;
408 + else if (reg == NAND_READ_LOCATION_2)
409 + snandc->regs->read_location1 = locreg_val;
410 + else if (reg == NAND_READ_LOCATION_3)
411 + snandc->regs->read_location3 = locreg_val;
412 +}
413 +
414 +static void qcom_spi_set_read_loc_last(struct qcom_nand_controller *snandc,
415 + int reg, int cw_offset, int read_size,
416 + int is_last_read_loc)
417 +{
418 + __le32 locreg_val;
419 + u32 val = (((cw_offset) << READ_LOCATION_OFFSET) |
420 + ((read_size) << READ_LOCATION_SIZE) | ((is_last_read_loc)
421 + << READ_LOCATION_LAST));
422 +
423 + locreg_val = cpu_to_le32(val);
424 +
425 + if (reg == NAND_READ_LOCATION_LAST_CW_0)
426 + snandc->regs->read_location_last0 = locreg_val;
427 + else if (reg == NAND_READ_LOCATION_LAST_CW_1)
428 + snandc->regs->read_location_last1 = locreg_val;
429 + else if (reg == NAND_READ_LOCATION_LAST_CW_2)
430 + snandc->regs->read_location_last2 = locreg_val;
431 + else if (reg == NAND_READ_LOCATION_LAST_CW_3)
432 + snandc->regs->read_location_last3 = locreg_val;
433 +}
434 +
435 +static struct qcom_nand_controller *nand_to_qcom_snand(struct nand_device *nand)
436 +{
437 + struct nand_ecc_engine *eng = nand->ecc.engine;
438 + struct qpic_spi_nand *qspi = ecceng_to_qspi(eng);
439 +
440 + return qspi->snandc;
441 +}
442 +
443 +static int qcom_spi_init(struct qcom_nand_controller *snandc)
444 +{
445 + u32 snand_cfg_val = 0x0;
446 + int ret;
447 +
448 + snand_cfg_val = FIELD_PREP(CLK_CNTR_INIT_VAL_VEC_MASK, CLK_CNTR_INIT_VAL_VEC) |
449 + FIELD_PREP(LOAD_CLK_CNTR_INIT_EN, 0) |
450 + FIELD_PREP(FEA_STATUS_DEV_ADDR_MASK, FEA_STATUS_DEV_ADDR) |
451 + FIELD_PREP(SPI_CFG, 0);
452 +
453 + snandc->regs->spi_cfg = cpu_to_le32(snand_cfg_val);
454 + snandc->regs->num_addr_cycle = cpu_to_le32(SPI_NUM_ADDR);
455 + snandc->regs->busy_wait_cnt = cpu_to_le32(SPI_WAIT_CNT);
456 +
457 + qcom_write_reg_dma(snandc, &snandc->regs->spi_cfg, NAND_FLASH_SPI_CFG, 1, 0);
458 +
459 + snand_cfg_val &= ~LOAD_CLK_CNTR_INIT_EN;
460 + snandc->regs->spi_cfg = cpu_to_le32(snand_cfg_val);
461 +
462 + qcom_write_reg_dma(snandc, &snandc->regs->spi_cfg, NAND_FLASH_SPI_CFG, 1, 0);
463 +
464 + qcom_write_reg_dma(snandc, &snandc->regs->num_addr_cycle, NAND_NUM_ADDR_CYCLES, 1, 0);
465 + qcom_write_reg_dma(snandc, &snandc->regs->busy_wait_cnt, NAND_BUSY_CHECK_WAIT_CNT, 1,
466 + NAND_BAM_NEXT_SGL);
467 +
468 + ret = qcom_submit_descs(snandc);
469 + if (ret) {
470 + dev_err(snandc->dev, "failure in submitting spi init descriptor\n");
471 + return ret;
472 + }
473 +
474 + return ret;
475 +}
476 +
477 +static int qcom_spi_ooblayout_ecc(struct mtd_info *mtd, int section,
478 + struct mtd_oob_region *oobregion)
479 +{
480 + struct nand_device *nand = mtd_to_nanddev(mtd);
481 + struct qcom_nand_controller *snandc = nand_to_qcom_snand(nand);
482 + struct qpic_ecc *qecc = snandc->qspi->ecc;
483 +
484 + if (section > 1)
485 + return -ERANGE;
486 +
487 + oobregion->length = qecc->ecc_bytes_hw + qecc->spare_bytes;
488 + oobregion->offset = mtd->oobsize - oobregion->length;
489 +
490 + return 0;
491 +}
492 +
493 +static int qcom_spi_ooblayout_free(struct mtd_info *mtd, int section,
494 + struct mtd_oob_region *oobregion)
495 +{
496 + struct nand_device *nand = mtd_to_nanddev(mtd);
497 + struct qcom_nand_controller *snandc = nand_to_qcom_snand(nand);
498 + struct qpic_ecc *qecc = snandc->qspi->ecc;
499 +
500 + if (section)
501 + return -ERANGE;
502 +
503 + oobregion->length = qecc->steps * 4;
504 + oobregion->offset = ((qecc->steps - 1) * qecc->bytes) + qecc->bbm_size;
505 +
506 + return 0;
507 +}
508 +
509 +static const struct mtd_ooblayout_ops qcom_spi_ooblayout = {
510 + .ecc = qcom_spi_ooblayout_ecc,
511 + .free = qcom_spi_ooblayout_free,
512 +};
513 +
514 +static int qcom_spi_ecc_init_ctx_pipelined(struct nand_device *nand)
515 +{
516 + struct qcom_nand_controller *snandc = nand_to_qcom_snand(nand);
517 + struct nand_ecc_props *conf = &nand->ecc.ctx.conf;
518 + struct mtd_info *mtd = nanddev_to_mtd(nand);
519 + int cwperpage, bad_block_byte;
520 + struct qpic_ecc *ecc_cfg;
521 +
522 + cwperpage = mtd->writesize / NANDC_STEP_SIZE;
523 + snandc->qspi->num_cw = cwperpage;
524 +
525 + ecc_cfg = kzalloc(sizeof(*ecc_cfg), GFP_KERNEL);
526 + if (!ecc_cfg)
527 + return -ENOMEM;
528 + snandc->qspi->oob_buf = kzalloc(mtd->writesize + mtd->oobsize,
529 + GFP_KERNEL);
530 + if (!snandc->qspi->oob_buf)
531 + return -ENOMEM;
532 +
533 + memset(snandc->qspi->oob_buf, 0xff, mtd->writesize + mtd->oobsize);
534 +
535 + nand->ecc.ctx.priv = ecc_cfg;
536 + snandc->qspi->mtd = mtd;
537 +
538 + ecc_cfg->ecc_bytes_hw = 7;
539 + ecc_cfg->spare_bytes = 4;
540 + ecc_cfg->bbm_size = 1;
541 + ecc_cfg->bch_enabled = true;
542 + ecc_cfg->bytes = ecc_cfg->ecc_bytes_hw + ecc_cfg->spare_bytes + ecc_cfg->bbm_size;
543 +
544 + ecc_cfg->steps = 4;
545 + ecc_cfg->strength = 4;
546 + ecc_cfg->step_size = 512;
547 + ecc_cfg->cw_data = 516;
548 + ecc_cfg->cw_size = ecc_cfg->cw_data + ecc_cfg->bytes;
549 + bad_block_byte = mtd->writesize - ecc_cfg->cw_size * (cwperpage - 1) + 1;
550 +
551 + mtd_set_ooblayout(mtd, &qcom_spi_ooblayout);
552 +
553 + ecc_cfg->cfg0 = FIELD_PREP(CW_PER_PAGE_MASK, (cwperpage - 1)) |
554 + FIELD_PREP(UD_SIZE_BYTES_MASK, ecc_cfg->cw_data) |
555 + FIELD_PREP(DISABLE_STATUS_AFTER_WRITE, 1) |
556 + FIELD_PREP(NUM_ADDR_CYCLES_MASK, 3) |
557 + FIELD_PREP(ECC_PARITY_SIZE_BYTES_RS, ecc_cfg->ecc_bytes_hw) |
558 + FIELD_PREP(STATUS_BFR_READ, 0) |
559 + FIELD_PREP(SET_RD_MODE_AFTER_STATUS, 1) |
560 + FIELD_PREP(SPARE_SIZE_BYTES_MASK, ecc_cfg->spare_bytes);
561 +
562 + ecc_cfg->cfg1 = FIELD_PREP(NAND_RECOVERY_CYCLES_MASK, 0) |
563 + FIELD_PREP(CS_ACTIVE_BSY, 0) |
564 + FIELD_PREP(BAD_BLOCK_BYTE_NUM_MASK, bad_block_byte) |
565 + FIELD_PREP(BAD_BLOCK_IN_SPARE_AREA, 0) |
566 + FIELD_PREP(WR_RD_BSY_GAP_MASK, 20) |
567 + FIELD_PREP(WIDE_FLASH, 0) |
568 + FIELD_PREP(ENABLE_BCH_ECC, ecc_cfg->bch_enabled);
569 +
570 + ecc_cfg->cfg0_raw = FIELD_PREP(CW_PER_PAGE_MASK, (cwperpage - 1)) |
571 + FIELD_PREP(NUM_ADDR_CYCLES_MASK, 3) |
572 + FIELD_PREP(UD_SIZE_BYTES_MASK, ecc_cfg->cw_size) |
573 + FIELD_PREP(SPARE_SIZE_BYTES_MASK, 0);
574 +
575 + ecc_cfg->cfg1_raw = FIELD_PREP(NAND_RECOVERY_CYCLES_MASK, 0) |
576 + FIELD_PREP(CS_ACTIVE_BSY, 0) |
577 + FIELD_PREP(BAD_BLOCK_BYTE_NUM_MASK, 17) |
578 + FIELD_PREP(BAD_BLOCK_IN_SPARE_AREA, 1) |
579 + FIELD_PREP(WR_RD_BSY_GAP_MASK, 20) |
580 + FIELD_PREP(WIDE_FLASH, 0) |
581 + FIELD_PREP(DEV0_CFG1_ECC_DISABLE, 1);
582 +
583 + ecc_cfg->ecc_bch_cfg = FIELD_PREP(ECC_CFG_ECC_DISABLE, !ecc_cfg->bch_enabled) |
584 + FIELD_PREP(ECC_SW_RESET, 0) |
585 + FIELD_PREP(ECC_NUM_DATA_BYTES_MASK, ecc_cfg->cw_data) |
586 + FIELD_PREP(ECC_FORCE_CLK_OPEN, 1) |
587 + FIELD_PREP(ECC_MODE_MASK, 0) |
588 + FIELD_PREP(ECC_PARITY_SIZE_BYTES_BCH_MASK, ecc_cfg->ecc_bytes_hw);
589 +
590 + ecc_cfg->ecc_buf_cfg = 0x203 << NUM_STEPS;
591 + ecc_cfg->clrflashstatus = FS_READY_BSY_N;
592 + ecc_cfg->clrreadstatus = 0xc0;
593 +
594 + conf->step_size = ecc_cfg->step_size;
595 + conf->strength = ecc_cfg->strength;
596 +
597 + snandc->regs->erased_cw_detect_cfg_clr = cpu_to_le32(CLR_ERASED_PAGE_DET);
598 + snandc->regs->erased_cw_detect_cfg_set = cpu_to_le32(SET_ERASED_PAGE_DET);
599 +
600 + dev_dbg(snandc->dev, "ECC strength: %u bits per %u bytes\n",
601 + ecc_cfg->strength, ecc_cfg->step_size);
602 +
603 + return 0;
604 +}
605 +
606 +static void qcom_spi_ecc_cleanup_ctx_pipelined(struct nand_device *nand)
607 +{
608 + struct qpic_ecc *ecc_cfg = nand_to_ecc_ctx(nand);
609 +
610 + kfree(ecc_cfg);
611 +}
612 +
613 +static int qcom_spi_ecc_prepare_io_req_pipelined(struct nand_device *nand,
614 + struct nand_page_io_req *req)
615 +{
616 + struct qcom_nand_controller *snandc = nand_to_qcom_snand(nand);
617 + struct qpic_ecc *ecc_cfg = nand_to_ecc_ctx(nand);
618 +
619 + snandc->qspi->ecc = ecc_cfg;
620 + snandc->qspi->raw_rw = false;
621 + snandc->qspi->oob_rw = false;
622 + snandc->qspi->page_rw = false;
623 +
624 + if (req->datalen)
625 + snandc->qspi->page_rw = true;
626 +
627 + if (req->ooblen)
628 + snandc->qspi->oob_rw = true;
629 +
630 + if (req->mode == MTD_OPS_RAW)
631 + snandc->qspi->raw_rw = true;
632 +
633 + return 0;
634 +}
635 +
636 +static int qcom_spi_ecc_finish_io_req_pipelined(struct nand_device *nand,
637 + struct nand_page_io_req *req)
638 +{
639 + struct qcom_nand_controller *snandc = nand_to_qcom_snand(nand);
640 + struct mtd_info *mtd = nanddev_to_mtd(nand);
641 +
642 + if (req->mode == MTD_OPS_RAW || req->type != NAND_PAGE_READ)
643 + return 0;
644 +
645 + if (snandc->qspi->ecc_stats.failed)
646 + mtd->ecc_stats.failed += snandc->qspi->ecc_stats.failed;
647 + else
648 + mtd->ecc_stats.corrected += snandc->qspi->ecc_stats.corrected;
649 +
650 + if (snandc->qspi->ecc_stats.failed)
651 + return -EBADMSG;
652 + else
653 + return snandc->qspi->ecc_stats.bitflips;
654 +}
655 +
656 +static struct nand_ecc_engine_ops qcom_spi_ecc_engine_ops_pipelined = {
657 + .init_ctx = qcom_spi_ecc_init_ctx_pipelined,
658 + .cleanup_ctx = qcom_spi_ecc_cleanup_ctx_pipelined,
659 + .prepare_io_req = qcom_spi_ecc_prepare_io_req_pipelined,
660 + .finish_io_req = qcom_spi_ecc_finish_io_req_pipelined,
661 +};
662 +
663 +/* helper to configure location register values */
664 +static void qcom_spi_set_read_loc(struct qcom_nand_controller *snandc, int cw, int reg,
665 + int cw_offset, int read_size, int is_last_read_loc)
666 +{
667 + int reg_base = NAND_READ_LOCATION_0;
668 + int num_cw = snandc->qspi->num_cw;
669 +
670 + if (cw == (num_cw - 1))
671 + reg_base = NAND_READ_LOCATION_LAST_CW_0;
672 +
673 + reg_base += reg * 4;
674 +
675 + if (cw == (num_cw - 1))
676 + return qcom_spi_set_read_loc_last(snandc, reg_base, cw_offset,
677 + read_size, is_last_read_loc);
678 + else
679 + return qcom_spi_set_read_loc_first(snandc, reg_base, cw_offset,
680 + read_size, is_last_read_loc);
681 +}
682 +
683 +static void
684 +qcom_spi_config_cw_read(struct qcom_nand_controller *snandc, bool use_ecc, int cw)
685 +{
686 + __le32 *reg = &snandc->regs->read_location0;
687 + int num_cw = snandc->qspi->num_cw;
688 +
689 + qcom_write_reg_dma(snandc, reg, NAND_READ_LOCATION_0, 4, NAND_BAM_NEXT_SGL);
690 + if (cw == (num_cw - 1)) {
691 + reg = &snandc->regs->read_location_last0;
692 + qcom_write_reg_dma(snandc, reg, NAND_READ_LOCATION_LAST_CW_0, 4,
693 + NAND_BAM_NEXT_SGL);
694 + }
695 +
696 + qcom_write_reg_dma(snandc, &snandc->regs->cmd, NAND_FLASH_CMD, 1, NAND_BAM_NEXT_SGL);
697 + qcom_write_reg_dma(snandc, &snandc->regs->exec, NAND_EXEC_CMD, 1, NAND_BAM_NEXT_SGL);
698 +
699 + qcom_read_reg_dma(snandc, NAND_FLASH_STATUS, 2, 0);
700 + qcom_read_reg_dma(snandc, NAND_ERASED_CW_DETECT_STATUS, 1,
701 + NAND_BAM_NEXT_SGL);
702 +}
703 +
704 +static int qcom_spi_block_erase(struct qcom_nand_controller *snandc)
705 +{
706 + struct qpic_ecc *ecc_cfg = snandc->qspi->ecc;
707 + int ret;
708 +
709 + snandc->buf_count = 0;
710 + snandc->buf_start = 0;
711 + qcom_clear_read_regs(snandc);
712 + qcom_clear_bam_transaction(snandc);
713 +
714 + snandc->regs->cmd = snandc->qspi->cmd;
715 + snandc->regs->addr0 = snandc->qspi->addr1;
716 + snandc->regs->addr1 = snandc->qspi->addr2;
717 + snandc->regs->cfg0 = cpu_to_le32(ecc_cfg->cfg0_raw & ~(7 << CW_PER_PAGE));
718 + snandc->regs->cfg1 = cpu_to_le32(ecc_cfg->cfg1_raw);
719 + snandc->regs->exec = cpu_to_le32(1);
720 +
721 + qcom_write_reg_dma(snandc, &snandc->regs->cmd, NAND_FLASH_CMD, 3, NAND_BAM_NEXT_SGL);
722 + qcom_write_reg_dma(snandc, &snandc->regs->cfg0, NAND_DEV0_CFG0, 2, NAND_BAM_NEXT_SGL);
723 + qcom_write_reg_dma(snandc, &snandc->regs->exec, NAND_EXEC_CMD, 1, NAND_BAM_NEXT_SGL);
724 +
725 + ret = qcom_submit_descs(snandc);
726 + if (ret) {
727 + dev_err(snandc->dev, "failure to erase block\n");
728 + return ret;
729 + }
730 +
731 + return 0;
732 +}
733 +
734 +static void qcom_spi_config_single_cw_page_read(struct qcom_nand_controller *snandc,
735 + bool use_ecc, int cw)
736 +{
737 + __le32 *reg = &snandc->regs->read_location0;
738 + int num_cw = snandc->qspi->num_cw;
739 +
740 + qcom_write_reg_dma(snandc, &snandc->regs->addr0, NAND_ADDR0, 2, 0);
741 + qcom_write_reg_dma(snandc, &snandc->regs->cfg0, NAND_DEV0_CFG0, 3, 0);
742 + qcom_write_reg_dma(snandc, &snandc->regs->erased_cw_detect_cfg_clr,
743 + NAND_ERASED_CW_DETECT_CFG, 1, 0);
744 + qcom_write_reg_dma(snandc, &snandc->regs->erased_cw_detect_cfg_set,
745 + NAND_ERASED_CW_DETECT_CFG, 1,
746 + NAND_ERASED_CW_SET | NAND_BAM_NEXT_SGL);
747 +
748 + if (cw == (num_cw - 1)) {
749 + reg = &snandc->regs->read_location_last0;
750 + qcom_write_reg_dma(snandc, reg, NAND_READ_LOCATION_LAST_CW_0, 4, NAND_BAM_NEXT_SGL);
751 + }
752 + qcom_write_reg_dma(snandc, &snandc->regs->cmd, NAND_FLASH_CMD, 1, NAND_BAM_NEXT_SGL);
753 + qcom_write_reg_dma(snandc, &snandc->regs->exec, NAND_EXEC_CMD, 1, NAND_BAM_NEXT_SGL);
754 +
755 + qcom_read_reg_dma(snandc, NAND_FLASH_STATUS, 1, 0);
756 +}
757 +
758 +static int qcom_spi_read_last_cw(struct qcom_nand_controller *snandc,
759 + const struct spi_mem_op *op)
760 +{
761 + struct qpic_ecc *ecc_cfg = snandc->qspi->ecc;
762 + struct mtd_info *mtd = snandc->qspi->mtd;
763 + int size, ret = 0;
764 + int col, bbpos;
765 + u32 cfg0, cfg1, ecc_bch_cfg;
766 + u32 num_cw = snandc->qspi->num_cw;
767 +
768 + qcom_clear_bam_transaction(snandc);
769 + qcom_clear_read_regs(snandc);
770 +
771 + size = ecc_cfg->cw_size;
772 + col = ecc_cfg->cw_size * (num_cw - 1);
773 +
774 + memset(snandc->data_buffer, 0xff, size);
775 + snandc->regs->addr0 = (snandc->qspi->addr1 | cpu_to_le32(col));
776 + snandc->regs->addr1 = snandc->qspi->addr2;
777 +
778 + cfg0 = (ecc_cfg->cfg0_raw & ~(7U << CW_PER_PAGE)) |
779 + 0 << CW_PER_PAGE;
780 + cfg1 = ecc_cfg->cfg1_raw;
781 + ecc_bch_cfg = 1 << ECC_CFG_ECC_DISABLE;
782 +
783 + snandc->regs->cmd = snandc->qspi->cmd;
784 + snandc->regs->cfg0 = cpu_to_le32(cfg0);
785 + snandc->regs->cfg1 = cpu_to_le32(cfg1);
786 + snandc->regs->ecc_bch_cfg = cpu_to_le32(ecc_bch_cfg);
787 + snandc->regs->clrflashstatus = cpu_to_le32(ecc_cfg->clrflashstatus);
788 + snandc->regs->clrreadstatus = cpu_to_le32(ecc_cfg->clrreadstatus);
789 + snandc->regs->exec = cpu_to_le32(1);
790 +
791 + qcom_spi_set_read_loc(snandc, num_cw - 1, 0, 0, ecc_cfg->cw_size, 1);
792 +
793 + qcom_spi_config_single_cw_page_read(snandc, false, num_cw - 1);
794 +
795 + qcom_read_data_dma(snandc, FLASH_BUF_ACC, snandc->data_buffer, size, 0);
796 +
797 + ret = qcom_submit_descs(snandc);
798 + if (ret) {
799 + dev_err(snandc->dev, "failed to read last cw\n");
800 + return ret;
801 + }
802 +
803 + qcom_nandc_dev_to_mem(snandc, true);
804 + u32 flash = le32_to_cpu(snandc->reg_read_buf[0]);
805 +
806 + if (flash & (FS_OP_ERR | FS_MPU_ERR))
807 + return -EIO;
808 +
809 + bbpos = mtd->writesize - ecc_cfg->cw_size * (num_cw - 1);
810 +
811 + if (snandc->data_buffer[bbpos] == 0xff)
812 + snandc->data_buffer[bbpos + 1] = 0xff;
813 + if (snandc->data_buffer[bbpos] != 0xff)
814 + snandc->data_buffer[bbpos + 1] = snandc->data_buffer[bbpos];
815 +
816 + memcpy(op->data.buf.in, snandc->data_buffer + bbpos, op->data.nbytes);
817 +
818 + return ret;
819 +}
820 +
821 +static int qcom_spi_check_error(struct qcom_nand_controller *snandc, u8 *data_buf, u8 *oob_buf)
822 +{
823 + struct snandc_read_status *buf;
824 + struct qpic_ecc *ecc_cfg = snandc->qspi->ecc;
825 + int i, num_cw = snandc->qspi->num_cw;
826 + bool flash_op_err = false, erased;
827 + unsigned int max_bitflips = 0;
828 + unsigned int uncorrectable_cws = 0;
829 +
830 + snandc->qspi->ecc_stats.failed = 0;
831 + snandc->qspi->ecc_stats.corrected = 0;
832 +
833 + qcom_nandc_dev_to_mem(snandc, true);
834 + buf = (struct snandc_read_status *)snandc->reg_read_buf;
835 +
836 + for (i = 0; i < num_cw; i++, buf++) {
837 + u32 flash, buffer, erased_cw;
838 + int data_len, oob_len;
839 +
840 + if (i == (num_cw - 1)) {
841 + data_len = NANDC_STEP_SIZE - ((num_cw - 1) << 2);
842 + oob_len = num_cw << 2;
843 + } else {
844 + data_len = ecc_cfg->cw_data;
845 + oob_len = 0;
846 + }
847 +
848 + flash = le32_to_cpu(buf->snandc_flash);
849 + buffer = le32_to_cpu(buf->snandc_buffer);
850 + erased_cw = le32_to_cpu(buf->snandc_erased_cw);
851 +
852 + if ((flash & FS_OP_ERR) && (buffer & BS_UNCORRECTABLE_BIT)) {
853 + if (ecc_cfg->bch_enabled)
854 + erased = (erased_cw & ERASED_CW) == ERASED_CW;
855 + else
856 + erased = false;
857 +
858 + if (!erased)
859 + uncorrectable_cws |= BIT(i);
860 +
861 + } else if (flash & (FS_OP_ERR | FS_MPU_ERR)) {
862 + flash_op_err = true;
863 + } else {
864 + unsigned int stat;
865 +
866 + stat = buffer & BS_CORRECTABLE_ERR_MSK;
867 + snandc->qspi->ecc_stats.corrected += stat;
868 + max_bitflips = max(max_bitflips, stat);
869 + }
870 +
871 + if (data_buf)
872 + data_buf += data_len;
873 + if (oob_buf)
874 + oob_buf += oob_len + ecc_cfg->bytes;
875 + }
876 +
877 + if (flash_op_err)
878 + return -EIO;
879 +
880 + if (!uncorrectable_cws)
881 + snandc->qspi->ecc_stats.bitflips = max_bitflips;
882 + else
883 + snandc->qspi->ecc_stats.failed++;
884 +
885 + return 0;
886 +}
887 +
888 +static int qcom_spi_check_raw_flash_errors(struct qcom_nand_controller *snandc, int cw_cnt)
889 +{
890 + int i;
891 +
892 + qcom_nandc_dev_to_mem(snandc, true);
893 +
894 + for (i = 0; i < cw_cnt; i++) {
895 + u32 flash = le32_to_cpu(snandc->reg_read_buf[i]);
896 +
897 + if (flash & (FS_OP_ERR | FS_MPU_ERR))
898 + return -EIO;
899 + }
900 +
901 + return 0;
902 +}
903 +
904 +static int qcom_spi_read_cw_raw(struct qcom_nand_controller *snandc, u8 *data_buf,
905 + u8 *oob_buf, int cw)
906 +{
907 + struct qpic_ecc *ecc_cfg = snandc->qspi->ecc;
908 + struct mtd_info *mtd = snandc->qspi->mtd;
909 + int data_size1, data_size2, oob_size1, oob_size2;
910 + int ret, reg_off = FLASH_BUF_ACC, read_loc = 0;
911 + int raw_cw = cw;
912 + u32 cfg0, cfg1, ecc_bch_cfg, num_cw = snandc->qspi->num_cw;
913 + int col;
914 +
915 + snandc->buf_count = 0;
916 + snandc->buf_start = 0;
917 + qcom_clear_read_regs(snandc);
918 + qcom_clear_bam_transaction(snandc);
919 + raw_cw = num_cw - 1;
920 +
921 + cfg0 = (ecc_cfg->cfg0_raw & ~(7U << CW_PER_PAGE)) |
922 + 0 << CW_PER_PAGE;
923 + cfg1 = ecc_cfg->cfg1_raw;
924 + ecc_bch_cfg = ECC_CFG_ECC_DISABLE;
925 +
926 + col = ecc_cfg->cw_size * cw;
927 +
928 + snandc->regs->addr0 = (snandc->qspi->addr1 | cpu_to_le32(col));
929 + snandc->regs->addr1 = snandc->qspi->addr2;
930 + snandc->regs->cmd = snandc->qspi->cmd;
931 + snandc->regs->cfg0 = cpu_to_le32(cfg0);
932 + snandc->regs->cfg1 = cpu_to_le32(cfg1);
933 + snandc->regs->ecc_bch_cfg = cpu_to_le32(ecc_bch_cfg);
934 + snandc->regs->clrflashstatus = cpu_to_le32(ecc_cfg->clrflashstatus);
935 + snandc->regs->clrreadstatus = cpu_to_le32(ecc_cfg->clrreadstatus);
936 + snandc->regs->exec = cpu_to_le32(1);
937 +
938 + qcom_spi_set_read_loc(snandc, raw_cw, 0, 0, ecc_cfg->cw_size, 1);
939 +
940 + qcom_write_reg_dma(snandc, &snandc->regs->addr0, NAND_ADDR0, 2, 0);
941 + qcom_write_reg_dma(snandc, &snandc->regs->cfg0, NAND_DEV0_CFG0, 3, 0);
942 + qcom_write_reg_dma(snandc, &snandc->regs->ecc_buf_cfg, NAND_EBI2_ECC_BUF_CFG, 1, 0);
943 +
944 + qcom_write_reg_dma(snandc, &snandc->regs->erased_cw_detect_cfg_clr,
945 + NAND_ERASED_CW_DETECT_CFG, 1, 0);
946 + qcom_write_reg_dma(snandc, &snandc->regs->erased_cw_detect_cfg_set,
947 + NAND_ERASED_CW_DETECT_CFG, 1,
948 + NAND_ERASED_CW_SET | NAND_BAM_NEXT_SGL);
949 +
950 + data_size1 = mtd->writesize - ecc_cfg->cw_size * (num_cw - 1);
951 + oob_size1 = ecc_cfg->bbm_size;
952 +
953 + if (cw == (num_cw - 1)) {
954 + data_size2 = NANDC_STEP_SIZE - data_size1 -
955 + ((num_cw - 1) * 4);
956 + oob_size2 = (num_cw * 4) + ecc_cfg->ecc_bytes_hw +
957 + ecc_cfg->spare_bytes;
958 + } else {
959 + data_size2 = ecc_cfg->cw_data - data_size1;
960 + oob_size2 = ecc_cfg->ecc_bytes_hw + ecc_cfg->spare_bytes;
961 + }
962 +
963 + qcom_spi_set_read_loc(snandc, cw, 0, read_loc, data_size1, 0);
964 + read_loc += data_size1;
965 +
966 + qcom_spi_set_read_loc(snandc, cw, 1, read_loc, oob_size1, 0);
967 + read_loc += oob_size1;
968 +
969 + qcom_spi_set_read_loc(snandc, cw, 2, read_loc, data_size2, 0);
970 + read_loc += data_size2;
971 +
972 + qcom_spi_set_read_loc(snandc, cw, 3, read_loc, oob_size2, 1);
973 +
974 + qcom_spi_config_cw_read(snandc, false, raw_cw);
975 +
976 + qcom_read_data_dma(snandc, reg_off, data_buf, data_size1, 0);
977 + reg_off += data_size1;
978 +
979 + qcom_read_data_dma(snandc, reg_off, oob_buf, oob_size1, 0);
980 + reg_off += oob_size1;
981 +
982 + qcom_read_data_dma(snandc, reg_off, data_buf + data_size1, data_size2, 0);
983 + reg_off += data_size2;
984 +
985 + qcom_read_data_dma(snandc, reg_off, oob_buf + oob_size1, oob_size2, 0);
986 +
987 + ret = qcom_submit_descs(snandc);
988 + if (ret) {
989 + dev_err(snandc->dev, "failure to read raw cw %d\n", cw);
990 + return ret;
991 + }
992 +
993 + return qcom_spi_check_raw_flash_errors(snandc, 1);
994 +}
995 +
996 +static int qcom_spi_read_page_raw(struct qcom_nand_controller *snandc,
997 + const struct spi_mem_op *op)
998 +{
999 + struct qpic_ecc *ecc_cfg = snandc->qspi->ecc;
1000 + u8 *data_buf = NULL, *oob_buf = NULL;
1001 + int ret, cw;
1002 + u32 num_cw = snandc->qspi->num_cw;
1003 +
1004 + if (snandc->qspi->page_rw)
1005 + data_buf = op->data.buf.in;
1006 +
1007 + oob_buf = snandc->qspi->oob_buf;
1008 + memset(oob_buf, 0xff, OOB_BUF_SIZE);
1009 +
1010 + for (cw = 0; cw < num_cw; cw++) {
1011 + ret = qcom_spi_read_cw_raw(snandc, data_buf, oob_buf, cw);
1012 + if (ret)
1013 + return ret;
1014 +
1015 + if (data_buf)
1016 + data_buf += ecc_cfg->cw_data;
1017 + if (oob_buf)
1018 + oob_buf += ecc_cfg->bytes;
1019 + }
1020 +
1021 + return 0;
1022 +}
1023 +
1024 +static int qcom_spi_read_page_ecc(struct qcom_nand_controller *snandc,
1025 + const struct spi_mem_op *op)
1026 +{
1027 + struct qpic_ecc *ecc_cfg = snandc->qspi->ecc;
1028 + u8 *data_buf = NULL, *data_buf_start, *oob_buf = NULL, *oob_buf_start;
1029 + int ret, i;
1030 + u32 cfg0, cfg1, ecc_bch_cfg, num_cw = snandc->qspi->num_cw;
1031 +
1032 + data_buf = op->data.buf.in;
1033 + data_buf_start = data_buf;
1034 +
1035 + oob_buf = snandc->qspi->oob_buf;
1036 + oob_buf_start = oob_buf;
1037 +
1038 + snandc->buf_count = 0;
1039 + snandc->buf_start = 0;
1040 + qcom_clear_read_regs(snandc);
1041 +
1042 + cfg0 = (ecc_cfg->cfg0 & ~(7U << CW_PER_PAGE)) |
1043 + (num_cw - 1) << CW_PER_PAGE;
1044 + cfg1 = ecc_cfg->cfg1;
1045 + ecc_bch_cfg = ecc_cfg->ecc_bch_cfg;
1046 +
1047 + snandc->regs->addr0 = snandc->qspi->addr1;
1048 + snandc->regs->addr1 = snandc->qspi->addr2;
1049 + snandc->regs->cmd = snandc->qspi->cmd;
1050 + snandc->regs->cfg0 = cpu_to_le32(cfg0);
1051 + snandc->regs->cfg1 = cpu_to_le32(cfg1);
1052 + snandc->regs->ecc_bch_cfg = cpu_to_le32(ecc_bch_cfg);
1053 + snandc->regs->clrflashstatus = cpu_to_le32(ecc_cfg->clrflashstatus);
1054 + snandc->regs->clrreadstatus = cpu_to_le32(ecc_cfg->clrreadstatus);
1055 + snandc->regs->exec = cpu_to_le32(1);
1056 +
1057 + qcom_spi_set_read_loc(snandc, 0, 0, 0, ecc_cfg->cw_data, 1);
1058 +
1059 + qcom_clear_bam_transaction(snandc);
1060 +
1061 + qcom_write_reg_dma(snandc, &snandc->regs->addr0, NAND_ADDR0, 2, 0);
1062 + qcom_write_reg_dma(snandc, &snandc->regs->cfg0, NAND_DEV0_CFG0, 3, 0);
1063 + qcom_write_reg_dma(snandc, &snandc->regs->erased_cw_detect_cfg_clr,
1064 + NAND_ERASED_CW_DETECT_CFG, 1, 0);
1065 + qcom_write_reg_dma(snandc, &snandc->regs->erased_cw_detect_cfg_set,
1066 + NAND_ERASED_CW_DETECT_CFG, 1,
1067 + NAND_ERASED_CW_SET | NAND_BAM_NEXT_SGL);
1068 +
1069 + for (i = 0; i < num_cw; i++) {
1070 + int data_size, oob_size;
1071 +
1072 + if (i == (num_cw - 1)) {
1073 + data_size = 512 - ((num_cw - 1) << 2);
1074 + oob_size = (num_cw << 2) + ecc_cfg->ecc_bytes_hw +
1075 + ecc_cfg->spare_bytes;
1076 + } else {
1077 + data_size = ecc_cfg->cw_data;
1078 + oob_size = ecc_cfg->ecc_bytes_hw + ecc_cfg->spare_bytes;
1079 + }
1080 +
1081 + if (data_buf && oob_buf) {
1082 + qcom_spi_set_read_loc(snandc, i, 0, 0, data_size, 0);
1083 + qcom_spi_set_read_loc(snandc, i, 1, data_size, oob_size, 1);
1084 + } else if (data_buf) {
1085 + qcom_spi_set_read_loc(snandc, i, 0, 0, data_size, 1);
1086 + } else {
1087 + qcom_spi_set_read_loc(snandc, i, 0, data_size, oob_size, 1);
1088 + }
1089 +
1090 + qcom_spi_config_cw_read(snandc, true, i);
1091 +
1092 + if (data_buf)
1093 + qcom_read_data_dma(snandc, FLASH_BUF_ACC, data_buf,
1094 + data_size, 0);
1095 + if (oob_buf) {
1096 + int j;
1097 +
1098 + for (j = 0; j < ecc_cfg->bbm_size; j++)
1099 + *oob_buf++ = 0xff;
1100 +
1101 + qcom_read_data_dma(snandc, FLASH_BUF_ACC + data_size,
1102 + oob_buf, oob_size, 0);
1103 + }
1104 +
1105 + if (data_buf)
1106 + data_buf += data_size;
1107 + if (oob_buf)
1108 + oob_buf += oob_size;
1109 + }
1110 +
1111 + ret = qcom_submit_descs(snandc);
1112 + if (ret) {
1113 + dev_err(snandc->dev, "failure to read page\n");
1114 + return ret;
1115 + }
1116 +
1117 + return qcom_spi_check_error(snandc, data_buf_start, oob_buf_start);
1118 +}
1119 +
1120 +static int qcom_spi_read_page_oob(struct qcom_nand_controller *snandc,
1121 + const struct spi_mem_op *op)
1122 +{
1123 + struct qpic_ecc *ecc_cfg = snandc->qspi->ecc;
1124 + u8 *data_buf = NULL, *data_buf_start, *oob_buf = NULL, *oob_buf_start;
1125 + int ret, i;
1126 + u32 cfg0, cfg1, ecc_bch_cfg, num_cw = snandc->qspi->num_cw;
1127 +
1128 + oob_buf = op->data.buf.in;
1129 + oob_buf_start = oob_buf;
1130 +
1131 + data_buf_start = data_buf;
1132 +
1133 + snandc->buf_count = 0;
1134 + snandc->buf_start = 0;
1135 + qcom_clear_read_regs(snandc);
1136 + qcom_clear_bam_transaction(snandc);
1137 +
1138 + cfg0 = (ecc_cfg->cfg0 & ~(7U << CW_PER_PAGE)) |
1139 + (num_cw - 1) << CW_PER_PAGE;
1140 + cfg1 = ecc_cfg->cfg1;
1141 + ecc_bch_cfg = ecc_cfg->ecc_bch_cfg;
1142 +
1143 + snandc->regs->addr0 = snandc->qspi->addr1;
1144 + snandc->regs->addr1 = snandc->qspi->addr2;
1145 + snandc->regs->cmd = snandc->qspi->cmd;
1146 + snandc->regs->cfg0 = cpu_to_le32(cfg0);
1147 + snandc->regs->cfg1 = cpu_to_le32(cfg1);
1148 + snandc->regs->ecc_bch_cfg = cpu_to_le32(ecc_bch_cfg);
1149 + snandc->regs->clrflashstatus = cpu_to_le32(ecc_cfg->clrflashstatus);
1150 + snandc->regs->clrreadstatus = cpu_to_le32(ecc_cfg->clrreadstatus);
1151 + snandc->regs->exec = cpu_to_le32(1);
1152 +
1153 + qcom_spi_set_read_loc(snandc, 0, 0, 0, ecc_cfg->cw_data, 1);
1154 +
1155 + qcom_write_reg_dma(snandc, &snandc->regs->addr0, NAND_ADDR0, 2, 0);
1156 + qcom_write_reg_dma(snandc, &snandc->regs->cfg0, NAND_DEV0_CFG0, 3, 0);
1157 + qcom_write_reg_dma(snandc, &snandc->regs->erased_cw_detect_cfg_clr,
1158 + NAND_ERASED_CW_DETECT_CFG, 1, 0);
1159 + qcom_write_reg_dma(snandc, &snandc->regs->erased_cw_detect_cfg_set,
1160 + NAND_ERASED_CW_DETECT_CFG, 1,
1161 + NAND_ERASED_CW_SET | NAND_BAM_NEXT_SGL);
1162 +
1163 + for (i = 0; i < num_cw; i++) {
1164 + int data_size, oob_size;
1165 +
1166 + if (i == (num_cw - 1)) {
1167 + data_size = NANDC_STEP_SIZE - ((num_cw - 1) << 2);
1168 + oob_size = (num_cw << 2) + ecc_cfg->ecc_bytes_hw +
1169 + ecc_cfg->spare_bytes;
1170 + } else {
1171 + data_size = ecc_cfg->cw_data;
1172 + oob_size = ecc_cfg->ecc_bytes_hw + ecc_cfg->spare_bytes;
1173 + }
1174 +
1175 + qcom_spi_set_read_loc(snandc, i, 0, data_size, oob_size, 1);
1176 +
1177 + qcom_spi_config_cw_read(snandc, true, i);
1178 +
1179 + if (oob_buf) {
1180 + int j;
1181 +
1182 + for (j = 0; j < ecc_cfg->bbm_size; j++)
1183 + *oob_buf++ = 0xff;
1184 +
1185 + qcom_read_data_dma(snandc, FLASH_BUF_ACC + data_size,
1186 + oob_buf, oob_size, 0);
1187 + }
1188 +
1189 + if (oob_buf)
1190 + oob_buf += oob_size;
1191 + }
1192 +
1193 + ret = qcom_submit_descs(snandc);
1194 + if (ret) {
1195 + dev_err(snandc->dev, "failure to read oob\n");
1196 + return ret;
1197 + }
1198 +
1199 + return qcom_spi_check_error(snandc, data_buf_start, oob_buf_start);
1200 +}
1201 +
1202 +static int qcom_spi_read_page(struct qcom_nand_controller *snandc,
1203 + const struct spi_mem_op *op)
1204 +{
1205 + if (snandc->qspi->page_rw && snandc->qspi->raw_rw)
1206 + return qcom_spi_read_page_raw(snandc, op);
1207 +
1208 + if (snandc->qspi->page_rw)
1209 + return qcom_spi_read_page_ecc(snandc, op);
1210 +
1211 + if (snandc->qspi->oob_rw && snandc->qspi->raw_rw)
1212 + return qcom_spi_read_last_cw(snandc, op);
1213 +
1214 + if (snandc->qspi->oob_rw)
1215 + return qcom_spi_read_page_oob(snandc, op);
1216 +
1217 + return 0;
1218 +}
1219 +
1220 +static void qcom_spi_config_page_write(struct qcom_nand_controller *snandc)
1221 +{
1222 + qcom_write_reg_dma(snandc, &snandc->regs->addr0, NAND_ADDR0, 2, 0);
1223 + qcom_write_reg_dma(snandc, &snandc->regs->cfg0, NAND_DEV0_CFG0, 3, 0);
1224 + qcom_write_reg_dma(snandc, &snandc->regs->ecc_buf_cfg, NAND_EBI2_ECC_BUF_CFG,
1225 + 1, NAND_BAM_NEXT_SGL);
1226 +}
1227 +
1228 +static void qcom_spi_config_cw_write(struct qcom_nand_controller *snandc)
1229 +{
1230 + qcom_write_reg_dma(snandc, &snandc->regs->cmd, NAND_FLASH_CMD, 1, NAND_BAM_NEXT_SGL);
1231 + qcom_write_reg_dma(snandc, &snandc->regs->exec, NAND_EXEC_CMD, 1, NAND_BAM_NEXT_SGL);
1232 + qcom_read_reg_dma(snandc, NAND_FLASH_STATUS, 1, NAND_BAM_NEXT_SGL);
1233 +
1234 + qcom_write_reg_dma(snandc, &snandc->regs->clrflashstatus, NAND_FLASH_STATUS, 1, 0);
1235 + qcom_write_reg_dma(snandc, &snandc->regs->clrreadstatus, NAND_READ_STATUS, 1,
1236 + NAND_BAM_NEXT_SGL);
1237 +}
1238 +
1239 +static int qcom_spi_program_raw(struct qcom_nand_controller *snandc,
1240 + const struct spi_mem_op *op)
1241 +{
1242 + struct qpic_ecc *ecc_cfg = snandc->qspi->ecc;
1243 + struct mtd_info *mtd = snandc->qspi->mtd;
1244 + u8 *data_buf = NULL, *oob_buf = NULL;
1245 + int i, ret;
1246 + int num_cw = snandc->qspi->num_cw;
1247 + u32 cfg0, cfg1, ecc_bch_cfg;
1248 +
1249 + cfg0 = (ecc_cfg->cfg0_raw & ~(7U << CW_PER_PAGE)) |
1250 + (num_cw - 1) << CW_PER_PAGE;
1251 + cfg1 = ecc_cfg->cfg1_raw;
1252 + ecc_bch_cfg = ECC_CFG_ECC_DISABLE;
1253 +
1254 + data_buf = snandc->qspi->data_buf;
1255 +
1256 + oob_buf = snandc->qspi->oob_buf;
1257 + memset(oob_buf, 0xff, OOB_BUF_SIZE);
1258 +
1259 + snandc->buf_count = 0;
1260 + snandc->buf_start = 0;
1261 + qcom_clear_read_regs(snandc);
1262 + qcom_clear_bam_transaction(snandc);
1263 +
1264 + snandc->regs->addr0 = snandc->qspi->addr1;
1265 + snandc->regs->addr1 = snandc->qspi->addr2;
1266 + snandc->regs->cmd = snandc->qspi->cmd;
1267 + snandc->regs->cfg0 = cpu_to_le32(cfg0);
1268 + snandc->regs->cfg1 = cpu_to_le32(cfg1);
1269 + snandc->regs->ecc_bch_cfg = cpu_to_le32(ecc_bch_cfg);
1270 + snandc->regs->clrflashstatus = cpu_to_le32(ecc_cfg->clrflashstatus);
1271 + snandc->regs->clrreadstatus = cpu_to_le32(ecc_cfg->clrreadstatus);
1272 + snandc->regs->exec = cpu_to_le32(1);
1273 +
1274 + qcom_spi_config_page_write(snandc);
1275 +
1276 + for (i = 0; i < num_cw; i++) {
1277 + int data_size1, data_size2, oob_size1, oob_size2;
1278 + int reg_off = FLASH_BUF_ACC;
1279 +
1280 + data_size1 = mtd->writesize - ecc_cfg->cw_size * (num_cw - 1);
1281 + oob_size1 = ecc_cfg->bbm_size;
1282 +
1283 + if (i == (num_cw - 1)) {
1284 + data_size2 = NANDC_STEP_SIZE - data_size1 -
1285 + ((num_cw - 1) << 2);
1286 + oob_size2 = (num_cw << 2) + ecc_cfg->ecc_bytes_hw +
1287 + ecc_cfg->spare_bytes;
1288 + } else {
1289 + data_size2 = ecc_cfg->cw_data - data_size1;
1290 + oob_size2 = ecc_cfg->ecc_bytes_hw + ecc_cfg->spare_bytes;
1291 + }
1292 +
1293 + qcom_write_data_dma(snandc, reg_off, data_buf, data_size1,
1294 + NAND_BAM_NO_EOT);
1295 + reg_off += data_size1;
1296 + data_buf += data_size1;
1297 +
1298 + qcom_write_data_dma(snandc, reg_off, oob_buf, oob_size1,
1299 + NAND_BAM_NO_EOT);
1300 + oob_buf += oob_size1;
1301 + reg_off += oob_size1;
1302 +
1303 + qcom_write_data_dma(snandc, reg_off, data_buf, data_size2,
1304 + NAND_BAM_NO_EOT);
1305 + reg_off += data_size2;
1306 + data_buf += data_size2;
1307 +
1308 + qcom_write_data_dma(snandc, reg_off, oob_buf, oob_size2, 0);
1309 + oob_buf += oob_size2;
1310 +
1311 + qcom_spi_config_cw_write(snandc);
1312 + }
1313 +
1314 + ret = qcom_submit_descs(snandc);
1315 + if (ret) {
1316 + dev_err(snandc->dev, "failure to write raw page\n");
1317 + return ret;
1318 + }
1319 +
1320 + return 0;
1321 +}
1322 +
1323 +static int qcom_spi_program_ecc(struct qcom_nand_controller *snandc,
1324 + const struct spi_mem_op *op)
1325 +{
1326 + struct qpic_ecc *ecc_cfg = snandc->qspi->ecc;
1327 + u8 *data_buf = NULL, *oob_buf = NULL;
1328 + int i, ret;
1329 + int num_cw = snandc->qspi->num_cw;
1330 + u32 cfg0, cfg1, ecc_bch_cfg, ecc_buf_cfg;
1331 +
1332 + cfg0 = (ecc_cfg->cfg0 & ~(7U << CW_PER_PAGE)) |
1333 + (num_cw - 1) << CW_PER_PAGE;
1334 + cfg1 = ecc_cfg->cfg1;
1335 + ecc_bch_cfg = ecc_cfg->ecc_bch_cfg;
1336 + ecc_buf_cfg = ecc_cfg->ecc_buf_cfg;
1337 +
1338 + if (snandc->qspi->data_buf)
1339 + data_buf = snandc->qspi->data_buf;
1340 +
1341 + oob_buf = snandc->qspi->oob_buf;
1342 +
1343 + snandc->buf_count = 0;
1344 + snandc->buf_start = 0;
1345 + qcom_clear_read_regs(snandc);
1346 + qcom_clear_bam_transaction(snandc);
1347 +
1348 + snandc->regs->addr0 = snandc->qspi->addr1;
1349 + snandc->regs->addr1 = snandc->qspi->addr2;
1350 + snandc->regs->cmd = snandc->qspi->cmd;
1351 + snandc->regs->cfg0 = cpu_to_le32(cfg0);
1352 + snandc->regs->cfg1 = cpu_to_le32(cfg1);
1353 + snandc->regs->ecc_bch_cfg = cpu_to_le32(ecc_bch_cfg);
1354 + snandc->regs->ecc_buf_cfg = cpu_to_le32(ecc_buf_cfg);
1355 + snandc->regs->exec = cpu_to_le32(1);
1356 +
1357 + qcom_spi_config_page_write(snandc);
1358 +
1359 + for (i = 0; i < num_cw; i++) {
1360 + int data_size, oob_size;
1361 +
1362 + if (i == (num_cw - 1)) {
1363 + data_size = NANDC_STEP_SIZE - ((num_cw - 1) << 2);
1364 + oob_size = (num_cw << 2) + ecc_cfg->ecc_bytes_hw +
1365 + ecc_cfg->spare_bytes;
1366 + } else {
1367 + data_size = ecc_cfg->cw_data;
1368 + oob_size = ecc_cfg->bytes;
1369 + }
1370 +
1371 + if (data_buf)
1372 + qcom_write_data_dma(snandc, FLASH_BUF_ACC, data_buf, data_size,
1373 + i == (num_cw - 1) ? NAND_BAM_NO_EOT : 0);
1374 +
1375 + if (i == (num_cw - 1)) {
1376 + if (oob_buf) {
1377 + oob_buf += ecc_cfg->bbm_size;
1378 + qcom_write_data_dma(snandc, FLASH_BUF_ACC + data_size,
1379 + oob_buf, oob_size, 0);
1380 + }
1381 + }
1382 +
1383 + qcom_spi_config_cw_write(snandc);
1384 +
1385 + if (data_buf)
1386 + data_buf += data_size;
1387 + if (oob_buf)
1388 + oob_buf += oob_size;
1389 + }
1390 +
1391 + ret = qcom_submit_descs(snandc);
1392 + if (ret) {
1393 + dev_err(snandc->dev, "failure to write page\n");
1394 + return ret;
1395 + }
1396 +
1397 + return 0;
1398 +}
1399 +
1400 +static int qcom_spi_program_oob(struct qcom_nand_controller *snandc,
1401 + const struct spi_mem_op *op)
1402 +{
1403 + struct qpic_ecc *ecc_cfg = snandc->qspi->ecc;
1404 + u8 *oob_buf = NULL;
1405 + int ret, col, data_size, oob_size;
1406 + int num_cw = snandc->qspi->num_cw;
1407 + u32 cfg0, cfg1, ecc_bch_cfg, ecc_buf_cfg;
1408 +
1409 + cfg0 = (ecc_cfg->cfg0 & ~(7U << CW_PER_PAGE)) |
1410 + (num_cw - 1) << CW_PER_PAGE;
1411 + cfg1 = ecc_cfg->cfg1;
1412 + ecc_bch_cfg = ecc_cfg->ecc_bch_cfg;
1413 + ecc_buf_cfg = ecc_cfg->ecc_buf_cfg;
1414 +
1415 + col = ecc_cfg->cw_size * (num_cw - 1);
1416 +
1417 + oob_buf = snandc->qspi->data_buf;
1418 +
1419 + snandc->buf_count = 0;
1420 + snandc->buf_start = 0;
1421 + qcom_clear_read_regs(snandc);
1422 + qcom_clear_bam_transaction(snandc);
1423 + snandc->regs->addr0 = (snandc->qspi->addr1 | cpu_to_le32(col));
1424 + snandc->regs->addr1 = snandc->qspi->addr2;
1425 + snandc->regs->cmd = snandc->qspi->cmd;
1426 + snandc->regs->cfg0 = cpu_to_le32(cfg0);
1427 + snandc->regs->cfg1 = cpu_to_le32(cfg1);
1428 + snandc->regs->ecc_bch_cfg = cpu_to_le32(ecc_bch_cfg);
1429 + snandc->regs->ecc_buf_cfg = cpu_to_le32(ecc_buf_cfg);
1430 + snandc->regs->exec = cpu_to_le32(1);
1431 +
1432 + /* calculate the data and oob size for the last codeword/step */
1433 + data_size = NANDC_STEP_SIZE - ((num_cw - 1) << 2);
1434 + oob_size = snandc->qspi->mtd->oobavail;
1435 +
1436 + memset(snandc->data_buffer, 0xff, ecc_cfg->cw_data);
1437 + /* override new oob content to last codeword */
1438 + mtd_ooblayout_get_databytes(snandc->qspi->mtd, snandc->data_buffer + data_size,
1439 + oob_buf, 0, snandc->qspi->mtd->oobavail);
1440 + qcom_spi_config_page_write(snandc);
1441 + qcom_write_data_dma(snandc, FLASH_BUF_ACC, snandc->data_buffer, data_size + oob_size, 0);
1442 + qcom_spi_config_cw_write(snandc);
1443 +
1444 + ret = qcom_submit_descs(snandc);
1445 + if (ret) {
1446 + dev_err(snandc->dev, "failure to write oob\n");
1447 + return ret;
1448 + }
1449 +
1450 + return 0;
1451 +}
1452 +
1453 +static int qcom_spi_program_execute(struct qcom_nand_controller *snandc,
1454 + const struct spi_mem_op *op)
1455 +{
1456 + if (snandc->qspi->page_rw && snandc->qspi->raw_rw)
1457 + return qcom_spi_program_raw(snandc, op);
1458 +
1459 + if (snandc->qspi->page_rw)
1460 + return qcom_spi_program_ecc(snandc, op);
1461 +
1462 + if (snandc->qspi->oob_rw)
1463 + return qcom_spi_program_oob(snandc, op);
1464 +
1465 + return 0;
1466 +}
1467 +
1468 +static int qcom_spi_cmd_mapping(struct qcom_nand_controller *snandc, u32 opcode)
1469 +{
1470 + int cmd = 0x0;
1471 +
1472 + switch (opcode) {
1473 + case SPINAND_RESET:
1474 + cmd = (SPI_WP | SPI_HOLD | SPI_TRANSFER_MODE_x1 | OP_RESET_DEVICE);
1475 + break;
1476 + case SPINAND_READID:
1477 + cmd = (SPI_WP | SPI_HOLD | SPI_TRANSFER_MODE_x1 | OP_FETCH_ID);
1478 + break;
1479 + case SPINAND_GET_FEATURE:
1480 + cmd = (SPI_TRANSFER_MODE_x1 | SPI_WP | SPI_HOLD | ACC_FEATURE);
1481 + break;
1482 + case SPINAND_SET_FEATURE:
1483 + cmd = (SPI_TRANSFER_MODE_x1 | SPI_WP | SPI_HOLD | ACC_FEATURE |
1484 + QPIC_SET_FEATURE);
1485 + break;
1486 + case SPINAND_READ:
1487 + if (snandc->qspi->raw_rw) {
1488 + cmd = (PAGE_ACC | LAST_PAGE | SPI_TRANSFER_MODE_x1 |
1489 + SPI_WP | SPI_HOLD | OP_PAGE_READ);
1490 + } else {
1491 + cmd = (PAGE_ACC | LAST_PAGE | SPI_TRANSFER_MODE_x1 |
1492 + SPI_WP | SPI_HOLD | OP_PAGE_READ_WITH_ECC);
1493 + }
1494 +
1495 + break;
1496 + case SPINAND_ERASE:
1497 + cmd = OP_BLOCK_ERASE | PAGE_ACC | LAST_PAGE | SPI_WP |
1498 + SPI_HOLD | SPI_TRANSFER_MODE_x1;
1499 + break;
1500 + case SPINAND_WRITE_EN:
1501 + cmd = SPINAND_WRITE_EN;
1502 + break;
1503 + case SPINAND_PROGRAM_EXECUTE:
1504 + cmd = (PAGE_ACC | LAST_PAGE | SPI_TRANSFER_MODE_x1 |
1505 + SPI_WP | SPI_HOLD | OP_PROGRAM_PAGE);
1506 + break;
1507 + case SPINAND_PROGRAM_LOAD:
1508 + cmd = SPINAND_PROGRAM_LOAD;
1509 + break;
1510 + default:
1511 + dev_err(snandc->dev, "Opcode not supported: %u\n", opcode);
1512 + return -EOPNOTSUPP;
1513 + }
1514 +
1515 + return cmd;
1516 +}
1517 +
1518 +static int qcom_spi_write_page(struct qcom_nand_controller *snandc,
1519 + const struct spi_mem_op *op)
1520 +{
1521 + int cmd;
1522 +
1523 + cmd = qcom_spi_cmd_mapping(snandc, op->cmd.opcode);
1524 + if (cmd < 0)
1525 + return cmd;
1526 +
1527 + if (op->cmd.opcode == SPINAND_PROGRAM_LOAD)
1528 + snandc->qspi->data_buf = (u8 *)op->data.buf.out;
1529 +
1530 + return 0;
1531 +}
1532 +
1533 +static int qcom_spi_send_cmdaddr(struct qcom_nand_controller *snandc,
1534 + const struct spi_mem_op *op)
1535 +{
1536 + struct qpic_snand_op s_op = {};
1537 + u32 cmd;
1538 + int ret, opcode;
1539 +
1540 + ret = qcom_spi_cmd_mapping(snandc, op->cmd.opcode);
1541 + if (ret < 0)
1542 + return ret;
1543 +
1544 + cmd = ret;
1545 +
1546 + s_op.cmd_reg = cmd;
1547 + s_op.addr1_reg = op->addr.val;
1548 + s_op.addr2_reg = 0;
1549 +
1550 + opcode = op->cmd.opcode;
1551 +
1552 + switch (opcode) {
1553 + case SPINAND_WRITE_EN:
1554 + return 0;
1555 + case SPINAND_PROGRAM_EXECUTE:
1556 + s_op.addr1_reg = op->addr.val << 16;
1557 + s_op.addr2_reg = op->addr.val >> 16 & 0xff;
1558 + snandc->qspi->addr1 = cpu_to_le32(s_op.addr1_reg);
1559 + snandc->qspi->addr2 = cpu_to_le32(s_op.addr2_reg);
1560 + snandc->qspi->cmd = cpu_to_le32(cmd);
1561 + return qcom_spi_program_execute(snandc, op);
1562 + case SPINAND_READ:
1563 + s_op.addr1_reg = (op->addr.val << 16);
1564 + s_op.addr2_reg = op->addr.val >> 16 & 0xff;
1565 + snandc->qspi->addr1 = cpu_to_le32(s_op.addr1_reg);
1566 + snandc->qspi->addr2 = cpu_to_le32(s_op.addr2_reg);
1567 + snandc->qspi->cmd = cpu_to_le32(cmd);
1568 + return 0;
1569 + case SPINAND_ERASE:
1570 + s_op.addr2_reg = (op->addr.val >> 16) & 0xffff;
1571 + s_op.addr1_reg = op->addr.val;
1572 + snandc->qspi->addr1 = cpu_to_le32(s_op.addr1_reg << 16);
1573 + snandc->qspi->addr2 = cpu_to_le32(s_op.addr2_reg);
1574 + snandc->qspi->cmd = cpu_to_le32(cmd);
1575 + qcom_spi_block_erase(snandc);
1576 + return 0;
1577 + default:
1578 + break;
1579 + }
1580 +
1581 + snandc->buf_count = 0;
1582 + snandc->buf_start = 0;
1583 + qcom_clear_read_regs(snandc);
1584 + qcom_clear_bam_transaction(snandc);
1585 +
1586 + snandc->regs->cmd = cpu_to_le32(s_op.cmd_reg);
1587 + snandc->regs->exec = cpu_to_le32(1);
1588 + snandc->regs->addr0 = cpu_to_le32(s_op.addr1_reg);
1589 + snandc->regs->addr1 = cpu_to_le32(s_op.addr2_reg);
1590 +
1591 + qcom_write_reg_dma(snandc, &snandc->regs->cmd, NAND_FLASH_CMD, 3, NAND_BAM_NEXT_SGL);
1592 + qcom_write_reg_dma(snandc, &snandc->regs->exec, NAND_EXEC_CMD, 1, NAND_BAM_NEXT_SGL);
1593 +
1594 + ret = qcom_submit_descs(snandc);
1595 + if (ret)
1596 + dev_err(snandc->dev, "failure in submitting cmd descriptor\n");
1597 +
1598 + return ret;
1599 +}
1600 +
1601 +static int qcom_spi_io_op(struct qcom_nand_controller *snandc, const struct spi_mem_op *op)
1602 +{
1603 + int ret, val, opcode;
1604 + bool copy = false, copy_ftr = false;
1605 +
1606 + ret = qcom_spi_send_cmdaddr(snandc, op);
1607 + if (ret)
1608 + return ret;
1609 +
1610 + snandc->buf_count = 0;
1611 + snandc->buf_start = 0;
1612 + qcom_clear_read_regs(snandc);
1613 + qcom_clear_bam_transaction(snandc);
1614 + opcode = op->cmd.opcode;
1615 +
1616 + switch (opcode) {
1617 + case SPINAND_READID:
1618 + snandc->buf_count = 4;
1619 + qcom_read_reg_dma(snandc, NAND_READ_ID, 1, NAND_BAM_NEXT_SGL);
1620 + copy = true;
1621 + break;
1622 + case SPINAND_GET_FEATURE:
1623 + snandc->buf_count = 4;
1624 + qcom_read_reg_dma(snandc, NAND_FLASH_FEATURES, 1, NAND_BAM_NEXT_SGL);
1625 + copy_ftr = true;
1626 + break;
1627 + case SPINAND_SET_FEATURE:
1628 + snandc->regs->flash_feature = cpu_to_le32(*(u32 *)op->data.buf.out);
1629 + qcom_write_reg_dma(snandc, &snandc->regs->flash_feature,
1630 + NAND_FLASH_FEATURES, 1, NAND_BAM_NEXT_SGL);
1631 + break;
1632 + case SPINAND_PROGRAM_EXECUTE:
1633 + case SPINAND_WRITE_EN:
1634 + case SPINAND_RESET:
1635 + case SPINAND_ERASE:
1636 + case SPINAND_READ:
1637 + return 0;
1638 + default:
1639 + return -EOPNOTSUPP;
1640 + }
1641 +
1642 + ret = qcom_submit_descs(snandc);
1643 + if (ret)
1644 + dev_err(snandc->dev, "failure in submitting descriptor for:%d\n", opcode);
1645 +
1646 + if (copy) {
1647 + qcom_nandc_dev_to_mem(snandc, true);
1648 + memcpy(op->data.buf.in, snandc->reg_read_buf, snandc->buf_count);
1649 + }
1650 +
1651 + if (copy_ftr) {
1652 + qcom_nandc_dev_to_mem(snandc, true);
1653 + val = le32_to_cpu(*(__le32 *)snandc->reg_read_buf);
1654 + val >>= 8;
1655 + memcpy(op->data.buf.in, &val, snandc->buf_count);
1656 + }
1657 +
1658 + return ret;
1659 +}
1660 +
1661 +static bool qcom_spi_is_page_op(const struct spi_mem_op *op)
1662 +{
1663 + if (op->addr.buswidth != 1 && op->addr.buswidth != 2 && op->addr.buswidth != 4)
1664 + return false;
1665 +
1666 + if (op->data.dir == SPI_MEM_DATA_IN) {
1667 + if (op->addr.buswidth == 4 && op->data.buswidth == 4)
1668 + return true;
1669 +
1670 + if (op->addr.nbytes == 2 && op->addr.buswidth == 1)
1671 + return true;
1672 +
1673 + } else if (op->data.dir == SPI_MEM_DATA_OUT) {
1674 + if (op->data.buswidth == 4)
1675 + return true;
1676 + if (op->addr.nbytes == 2 && op->addr.buswidth == 1)
1677 + return true;
1678 + }
1679 +
1680 + return false;
1681 +}
1682 +
1683 +static bool qcom_spi_supports_op(struct spi_mem *mem, const struct spi_mem_op *op)
1684 +{
1685 + if (!spi_mem_default_supports_op(mem, op))
1686 + return false;
1687 +
1688 + if (op->cmd.nbytes != 1 || op->cmd.buswidth != 1)
1689 + return false;
1690 +
1691 + if (qcom_spi_is_page_op(op))
1692 + return true;
1693 +
1694 + return ((!op->addr.nbytes || op->addr.buswidth == 1) &&
1695 + (!op->dummy.nbytes || op->dummy.buswidth == 1) &&
1696 + (!op->data.nbytes || op->data.buswidth == 1));
1697 +}
1698 +
1699 +static int qcom_spi_exec_op(struct spi_mem *mem, const struct spi_mem_op *op)
1700 +{
1701 + struct qcom_nand_controller *snandc = spi_controller_get_devdata(mem->spi->controller);
1702 +
1703 + dev_dbg(snandc->dev, "OP %02x ADDR %08llX@%d:%u DATA %d:%u", op->cmd.opcode,
1704 + op->addr.val, op->addr.buswidth, op->addr.nbytes,
1705 + op->data.buswidth, op->data.nbytes);
1706 +
1707 + if (qcom_spi_is_page_op(op)) {
1708 + if (op->data.dir == SPI_MEM_DATA_IN)
1709 + return qcom_spi_read_page(snandc, op);
1710 + if (op->data.dir == SPI_MEM_DATA_OUT)
1711 + return qcom_spi_write_page(snandc, op);
1712 + } else {
1713 + return qcom_spi_io_op(snandc, op);
1714 + }
1715 +
1716 + return 0;
1717 +}
1718 +
1719 +static const struct spi_controller_mem_ops qcom_spi_mem_ops = {
1720 + .supports_op = qcom_spi_supports_op,
1721 + .exec_op = qcom_spi_exec_op,
1722 +};
1723 +
1724 +static const struct spi_controller_mem_caps qcom_spi_mem_caps = {
1725 + .ecc = true,
1726 +};
1727 +
1728 +static int qcom_spi_probe(struct platform_device *pdev)
1729 +{
1730 + struct device *dev = &pdev->dev;
1731 + struct spi_controller *ctlr;
1732 + struct qcom_nand_controller *snandc;
1733 + struct qpic_spi_nand *qspi;
1734 + struct qpic_ecc *ecc;
1735 + struct resource *res;
1736 + const void *dev_data;
1737 + int ret;
1738 +
1739 + ecc = devm_kzalloc(dev, sizeof(*ecc), GFP_KERNEL);
1740 + if (!ecc)
1741 + return -ENOMEM;
1742 +
1743 + qspi = devm_kzalloc(dev, sizeof(*qspi), GFP_KERNEL);
1744 + if (!qspi)
1745 + return -ENOMEM;
1746 +
1747 + ctlr = __devm_spi_alloc_controller(dev, sizeof(*snandc), false);
1748 + if (!ctlr)
1749 + return -ENOMEM;
1750 +
1751 + platform_set_drvdata(pdev, ctlr);
1752 +
1753 + snandc = spi_controller_get_devdata(ctlr);
1754 + qspi->snandc = snandc;
1755 +
1756 + snandc->dev = dev;
1757 + snandc->qspi = qspi;
1758 + snandc->qspi->ctlr = ctlr;
1759 + snandc->qspi->ecc = ecc;
1760 +
1761 + dev_data = of_device_get_match_data(dev);
1762 + if (!dev_data) {
1763 + dev_err(&pdev->dev, "failed to get device data\n");
1764 + return -ENODEV;
1765 + }
1766 +
1767 + snandc->props = dev_data;
1768 + snandc->dev = &pdev->dev;
1769 +
1770 + snandc->core_clk = devm_clk_get(dev, "core");
1771 + if (IS_ERR(snandc->core_clk))
1772 + return PTR_ERR(snandc->core_clk);
1773 +
1774 + snandc->aon_clk = devm_clk_get(dev, "aon");
1775 + if (IS_ERR(snandc->aon_clk))
1776 + return PTR_ERR(snandc->aon_clk);
1777 +
1778 + snandc->qspi->iomacro_clk = devm_clk_get(dev, "iom");
1779 + if (IS_ERR(snandc->qspi->iomacro_clk))
1780 + return PTR_ERR(snandc->qspi->iomacro_clk);
1781 +
1782 + snandc->base = devm_platform_get_and_ioremap_resource(pdev, 0, &res);
1783 + if (IS_ERR(snandc->base))
1784 + return PTR_ERR(snandc->base);
1785 +
1786 + snandc->base_phys = res->start;
1787 + snandc->base_dma = dma_map_resource(dev, res->start, resource_size(res),
1788 + DMA_BIDIRECTIONAL, 0);
1789 + if (dma_mapping_error(dev, snandc->base_dma))
1790 + return -ENXIO;
1791 +
1792 + ret = clk_prepare_enable(snandc->core_clk);
1793 + if (ret)
1794 + goto err_dis_core_clk;
1795 +
1796 + ret = clk_prepare_enable(snandc->aon_clk);
1797 + if (ret)
1798 + goto err_dis_aon_clk;
1799 +
1800 + ret = clk_prepare_enable(snandc->qspi->iomacro_clk);
1801 + if (ret)
1802 + goto err_dis_iom_clk;
1803 +
1804 + ret = qcom_nandc_alloc(snandc);
1805 + if (ret)
1806 + goto err_snand_alloc;
1807 +
1808 + ret = qcom_spi_init(snandc);
1809 + if (ret)
1810 + goto err_spi_init;
1811 +
1812 + /* setup ECC engine */
1813 + snandc->qspi->ecc_eng.dev = &pdev->dev;
1814 + snandc->qspi->ecc_eng.integration = NAND_ECC_ENGINE_INTEGRATION_PIPELINED;
1815 + snandc->qspi->ecc_eng.ops = &qcom_spi_ecc_engine_ops_pipelined;
1816 + snandc->qspi->ecc_eng.priv = snandc;
1817 +
1818 + ret = nand_ecc_register_on_host_hw_engine(&snandc->qspi->ecc_eng);
1819 + if (ret) {
1820 + dev_err(&pdev->dev, "failed to register ecc engine:%d\n", ret);
1821 + goto err_spi_init;
1822 + }
1823 +
1824 + ctlr->num_chipselect = QPIC_QSPI_NUM_CS;
1825 + ctlr->mem_ops = &qcom_spi_mem_ops;
1826 + ctlr->mem_caps = &qcom_spi_mem_caps;
1827 + ctlr->dev.of_node = pdev->dev.of_node;
1828 + ctlr->mode_bits = SPI_TX_DUAL | SPI_RX_DUAL |
1829 + SPI_TX_QUAD | SPI_RX_QUAD;
1830 +
1831 + ret = spi_register_controller(ctlr);
1832 + if (ret) {
1833 + dev_err(&pdev->dev, "spi_register_controller failed.\n");
1834 + goto err_spi_init;
1835 + }
1836 +
1837 + return 0;
1838 +
1839 +err_spi_init:
1840 + qcom_nandc_unalloc(snandc);
1841 +err_snand_alloc:
1842 + clk_disable_unprepare(snandc->qspi->iomacro_clk);
1843 +err_dis_iom_clk:
1844 + clk_disable_unprepare(snandc->aon_clk);
1845 +err_dis_aon_clk:
1846 + clk_disable_unprepare(snandc->core_clk);
1847 +err_dis_core_clk:
1848 + dma_unmap_resource(dev, res->start, resource_size(res),
1849 + DMA_BIDIRECTIONAL, 0);
1850 + return ret;
1851 +}
1852 +
1853 +static void qcom_spi_remove(struct platform_device *pdev)
1854 +{
1855 + struct spi_controller *ctlr = platform_get_drvdata(pdev);
1856 + struct qcom_nand_controller *snandc = spi_controller_get_devdata(ctlr);
1857 + struct resource *res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
1858 +
1859 + spi_unregister_controller(ctlr);
1860 +
1861 + qcom_nandc_unalloc(snandc);
1862 +
1863 + clk_disable_unprepare(snandc->aon_clk);
1864 + clk_disable_unprepare(snandc->core_clk);
1865 + clk_disable_unprepare(snandc->qspi->iomacro_clk);
1866 +
1867 + dma_unmap_resource(&pdev->dev, snandc->base_dma, resource_size(res),
1868 + DMA_BIDIRECTIONAL, 0);
1869 +}
1870 +
1871 +static const struct qcom_nandc_props ipq9574_snandc_props = {
1872 + .dev_cmd_reg_start = 0x7000,
1873 + .supports_bam = true,
1874 +};
1875 +
1876 +static const struct of_device_id qcom_snandc_of_match[] = {
1877 + {
1878 + .compatible = "qcom,ipq9574-snand",
1879 + .data = &ipq9574_snandc_props,
1880 + },
1881 + {}
1882 +}
1883 +MODULE_DEVICE_TABLE(of, qcom_snandc_of_match);
1884 +
1885 +static struct platform_driver qcom_spi_driver = {
1886 + .driver = {
1887 + .name = "qcom_snand",
1888 + .of_match_table = qcom_snandc_of_match,
1889 + },
1890 + .probe = qcom_spi_probe,
1891 + .remove_new = qcom_spi_remove,
1892 +};
1893 +module_platform_driver(qcom_spi_driver);
1894 +
1895 +MODULE_DESCRIPTION("SPI driver for QPIC QSPI cores");
1896 +MODULE_AUTHOR("Md Sadre Alam <quic_mdalam@quicinc.com>");
1897 +MODULE_LICENSE("GPL");
1898 +
1899 diff --git a/include/linux/mtd/nand-qpic-common.h b/include/linux/mtd/nand-qpic-common.h
1900 index e79c79775eb8..7dba89654d6c 100644
1901 --- a/include/linux/mtd/nand-qpic-common.h
1902 +++ b/include/linux/mtd/nand-qpic-common.h
1903 @@ -322,6 +322,10 @@ struct nandc_regs {
1904 __le32 read_location_last1;
1905 __le32 read_location_last2;
1906 __le32 read_location_last3;
1907 + __le32 spi_cfg;
1908 + __le32 num_addr_cycle;
1909 + __le32 busy_wait_cnt;
1910 + __le32 flash_feature;
1911
1912 __le32 erased_cw_detect_cfg_clr;
1913 __le32 erased_cw_detect_cfg_set;
1914 @@ -336,6 +340,7 @@ struct nandc_regs {
1915 *
1916 * @core_clk: controller clock
1917 * @aon_clk: another controller clock
1918 + * @iomacro_clk: io macro clock
1919 *
1920 * @regs: a contiguous chunk of memory for DMA register
1921 * writes. contains the register values to be
1922 @@ -345,6 +350,7 @@ struct nandc_regs {
1923 * initialized via DT match data
1924 *
1925 * @controller: base controller structure
1926 + * @qspi: qpic spi structure
1927 * @host_list: list containing all the chips attached to the
1928 * controller
1929 *
1930 @@ -389,6 +395,7 @@ struct qcom_nand_controller {
1931 const struct qcom_nandc_props *props;
1932
1933 struct nand_controller *controller;
1934 + struct qpic_spi_nand *qspi;
1935 struct list_head host_list;
1936
1937 union {
1938 --
1939 2.34.1
Staging tree of Stijn Tintel